Minki/linux
1
0
Fork 1
mirror of https://github.com/torvalds/linux.git synced 2024-11-10 14:11:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
9553636b57
linux/drivers/pci/ats.c

509 lines
11 KiB
C
Raw Normal View History

PCI: Add SPDX GPL-2.0 when no license was specified b24413180f56 ("License cleanup: add SPDX GPL-2.0 license identifier to files with no license") added SPDX GPL-2.0 to several PCI files that previously contained no license information. Add SPDX GPL-2.0 to all other PCI files that did not contain any license information and hence were under the default GPL version 2 license of the kernel. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-01-26 17:45:16 +00:00
// SPDX-License-Identifier: GPL-2.0
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
/*
PCI: Tidy comments Remove pointless comments that tell us the file name, remove blank line comments, follow multi-line comment conventions. No functional change intended. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2018-03-09 22:36:33 +00:00
* PCI Express I/O Virtualization (IOV) support
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
* Address Translation Service 1.0
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
* Page Request Interface added by Joerg Roedel <joerg.roedel@amd.com>
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
* PASID support added by Joerg Roedel <joerg.roedel@amd.com>
PCI: Tidy comments Remove pointless comments that tell us the file name, remove blank line comments, follow multi-line comment conventions. No functional change intended. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2018-03-09 22:36:33 +00:00
*
* Copyright (C) 2009 Intel Corporation, Yu Zhao <yu.zhao@intel.com>
* Copyright (C) 2011 Advanced Micro Devices,
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
*/
PCI/ATS: Use FIELD_GET() Use FIELD_GET() to remove dependences on the field position, i.e., the shift value. No functional change intended. Link: https://lore.kernel.org/r/20231010204436.1000644-6-helgaas@kernel.org Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
2023-10-10 20:44:31 +00:00
#include <linux/bitfield.h>
pci: Fix files needing export.h for EXPORT_SYMBOL/THIS_MODULE They were implicitly getting it from device.h --> module.h but we want to clean that up. So add the minimal header for these macros. Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2011-05-27 13:37:25 +00:00
#include <linux/export.h>
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
#include <linux/pci-ats.h>
#include <linux/pci.h>
PCI: fix ats compile failure I get this compile failure on parisc: drivers/pci/ats.c: In function 'ats_alloc_one': drivers/pci/ats.c:29: error: implicit declaration of function 'kzalloc' drivers/pci/ats.c:29: warning: assignment makes pointer from integer without a cast drivers/pci/ats.c: In function 'ats_free_one': drivers/pci/ats.c:45: error: implicit declaration of function 'kfree' Because ats.c is missing linux/slab.h as an include. This patch fixes it Signed-off-by: James Bottomley <JBottomley@Parallels.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-11-29 19:20:23 +00:00
#include <linux/slab.h>
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
#include "pci.h"
PCI: Inline the ATS setup code into pci_ats_init() The ATS setup code in ats_alloc_one() is only used by pci_ats_init(), so inline it there. No functional change. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:35:18 +00:00
void pci_ats_init(struct pci_dev *dev)
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
{
int pos;
PCI: Add "pci=noats" boot parameter Adds a "pci=noats" boot parameter. When supplied, all ATS related functions fail immediately and the IOMMU is configured to not use device-IOTLB. Any function that checks for ATS capabilities directly against the devices should also check this flag. Currently, such functions exist only in IOMMU drivers, and they are covered by this patch. The motivation behind this patch is the existence of malicious devices. Lots of research has been done about how to use the IOMMU as protection from such devices. When ATS is supported, any I/O device can access any physical address by faking device-IOTLB entries. Adding the ability to ignore these entries lets sysadmins enhance system security. Signed-off-by: Gil Kupfer <gilkup@cs.technion.ac.il> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Joerg Roedel <jroedel@suse.de>
2018-05-10 22:56:02 +00:00
if (pci_ats_disabled())
return;
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ATS);
if (!pos)
PCI: Allocate ATS struct during enumeration Previously, we allocated pci_ats structures when an IOMMU driver called pci_enable_ats(). An SR-IOV VF shares the STU setting with its PF, so when enabling ATS on the VF, we allocated a pci_ats struct for the PF if it didn't already have one. We held the sriov->lock to serialize threads concurrently enabling ATS on several VFS so only one would allocate the PF pci_ats. Gregor reported a deadlock here: pci_enable_sriov sriov_enable virtfn_add mutex_lock(dev->sriov->lock) # acquire sriov->lock pci_device_add device_add BUS_NOTIFY_ADD_DEVICE notifier chain iommu_bus_notifier amd_iommu_add_device # iommu_ops.add_device init_iommu_group iommu_group_get_for_dev iommu_group_add_device __iommu_attach_device amd_iommu_attach_device # iommu_ops.attach_device attach_device pci_enable_ats mutex_lock(dev->sriov->lock) # deadlock There's no reason to delay allocating the pci_ats struct, and if we allocate it for each device at enumeration-time, there's no need for locking in pci_enable_ats(). Allocate pci_ats struct during enumeration, when we initialize other capabilities. Note that this implementation requires ATS to be enabled on the PF first, before on any of the VFs because the PF controls the STU for all the VFs. Link: http://permalink.gmane.org/gmane.linux.kernel.iommu/9433 Reported-by: Gregor Dick <gdick@solarflare.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:05:46 +00:00
return;
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
PCI: Embed ATS info directly into struct pci_dev The pci_ats struct is small and will get smaller, so I don't think it's worth allocating it separately from the pci_dev struct. Embed the ATS fields directly into struct pci_dev. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:15:19 +00:00
dev->ats_cap = pos;
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
}
PCI/ATS: Only enable ATS for trusted devices Add pci_ats_supported(), which checks whether a device has an ATS capability, and whether it is trusted. A device is untrusted if it is plugged into an external-facing port such as Thunderbolt and could be spoofing an existing device to exploit weaknesses in the IOMMU configuration. PCIe ATS is one such weaknesses since it allows endpoints to cache IOMMU translations and emit transactions with 'Translated' Address Type (10b) that partially bypass the IOMMU translation. The SMMUv3 and VT-d IOMMU drivers already disallow ATS and transactions with 'Translated' Address Type for untrusted devices. Add the check to pci_enable_ats() to let other drivers (AMD IOMMU for now) benefit from it. By checking ats_cap, the pci_ats_supported() helper also returns whether ATS was globally disabled with pci=noats, and could later include more things, for example whether the whole PCIe hierarchy down to the endpoint supports ATS. Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Reviewed-by: Joerg Roedel <jroedel@suse.de> Acked-by: Bjorn Helgaas <bhelgaas@google.com> Link: https://lore.kernel.org/r/20200520152201.3309416-2-jean-philippe@linaro.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-20 15:22:00 +00:00
/**
* pci_ats_supported - check if the device can use ATS
* @dev: the PCI device
*
* Returns true if the device supports ATS and is allowed to use it, false
* otherwise.
*/
bool pci_ats_supported(struct pci_dev *dev)
{
if (!dev->ats_cap)
return false;
return (dev->untrusted == 0);
}
EXPORT_SYMBOL_GPL(pci_ats_supported);
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
/**
* pci_enable_ats - enable the ATS capability
* @dev: the PCI device
* @ps: the IOMMU page shift
*
* Returns 0 on success, or negative on failure.
*/
int pci_enable_ats(struct pci_dev *dev, int ps)
{
u16 ctrl;
PCI: Use pci_physfn() rather than looking up physfn by hand Use the pci_physfn() helper rather than looking up physfn by hand. No functional change. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:38:13 +00:00
struct pci_dev *pdev;
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
PCI/ATS: Only enable ATS for trusted devices Add pci_ats_supported(), which checks whether a device has an ATS capability, and whether it is trusted. A device is untrusted if it is plugged into an external-facing port such as Thunderbolt and could be spoofing an existing device to exploit weaknesses in the IOMMU configuration. PCIe ATS is one such weaknesses since it allows endpoints to cache IOMMU translations and emit transactions with 'Translated' Address Type (10b) that partially bypass the IOMMU translation. The SMMUv3 and VT-d IOMMU drivers already disallow ATS and transactions with 'Translated' Address Type for untrusted devices. Add the check to pci_enable_ats() to let other drivers (AMD IOMMU for now) benefit from it. By checking ats_cap, the pci_ats_supported() helper also returns whether ATS was globally disabled with pci=noats, and could later include more things, for example whether the whole PCIe hierarchy down to the endpoint supports ATS. Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Reviewed-by: Joerg Roedel <jroedel@suse.de> Acked-by: Bjorn Helgaas <bhelgaas@google.com> Link: https://lore.kernel.org/r/20200520152201.3309416-2-jean-philippe@linaro.org Signed-off-by: Joerg Roedel <jroedel@suse.de>
2020-05-20 15:22:00 +00:00
if (!pci_ats_supported(dev))
PCI: Allocate ATS struct during enumeration Previously, we allocated pci_ats structures when an IOMMU driver called pci_enable_ats(). An SR-IOV VF shares the STU setting with its PF, so when enabling ATS on the VF, we allocated a pci_ats struct for the PF if it didn't already have one. We held the sriov->lock to serialize threads concurrently enabling ATS on several VFS so only one would allocate the PF pci_ats. Gregor reported a deadlock here: pci_enable_sriov sriov_enable virtfn_add mutex_lock(dev->sriov->lock) # acquire sriov->lock pci_device_add device_add BUS_NOTIFY_ADD_DEVICE notifier chain iommu_bus_notifier amd_iommu_add_device # iommu_ops.add_device init_iommu_group iommu_group_get_for_dev iommu_group_add_device __iommu_attach_device amd_iommu_attach_device # iommu_ops.attach_device attach_device pci_enable_ats mutex_lock(dev->sriov->lock) # deadlock There's no reason to delay allocating the pci_ats struct, and if we allocate it for each device at enumeration-time, there's no need for locking in pci_enable_ats(). Allocate pci_ats struct during enumeration, when we initialize other capabilities. Note that this implementation requires ATS to be enabled on the PF first, before on any of the VFs because the PF controls the STU for all the VFs. Link: http://permalink.gmane.org/gmane.linux.kernel.iommu/9433 Reported-by: Gregor Dick <gdick@solarflare.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:05:46 +00:00
return -EINVAL;
PCI: Remove pci_ats_enabled() Remove pci_ats_enabled(). There are no callers outside the ATS code itself. We don't need to check ats_cap, because if we don't find an ATS capability, we'll never set ats_enabled. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-20 14:23:37 +00:00
if (WARN_ON(dev->ats_enabled))
PCI: Clean up ATS error handling There's no need to BUG() if we enable ATS when it's already enabled. We don't need to BUG() when disabling ATS on a device that doesn't support ATS or if it's already disabled. If ATS is enabled, certainly we found an ATS capability in the past, so it should still be there now. Clean up these error paths. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:43:27 +00:00
return -EBUSY;
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
if (ps < PCI_ATS_MIN_STU)
return -EINVAL;
PCI: Allocate ATS struct during enumeration Previously, we allocated pci_ats structures when an IOMMU driver called pci_enable_ats(). An SR-IOV VF shares the STU setting with its PF, so when enabling ATS on the VF, we allocated a pci_ats struct for the PF if it didn't already have one. We held the sriov->lock to serialize threads concurrently enabling ATS on several VFS so only one would allocate the PF pci_ats. Gregor reported a deadlock here: pci_enable_sriov sriov_enable virtfn_add mutex_lock(dev->sriov->lock) # acquire sriov->lock pci_device_add device_add BUS_NOTIFY_ADD_DEVICE notifier chain iommu_bus_notifier amd_iommu_add_device # iommu_ops.add_device init_iommu_group iommu_group_get_for_dev iommu_group_add_device __iommu_attach_device amd_iommu_attach_device # iommu_ops.attach_device attach_device pci_enable_ats mutex_lock(dev->sriov->lock) # deadlock There's no reason to delay allocating the pci_ats struct, and if we allocate it for each device at enumeration-time, there's no need for locking in pci_enable_ats(). Allocate pci_ats struct during enumeration, when we initialize other capabilities. Note that this implementation requires ATS to be enabled on the PF first, before on any of the VFs because the PF controls the STU for all the VFs. Link: http://permalink.gmane.org/gmane.linux.kernel.iommu/9433 Reported-by: Gregor Dick <gdick@solarflare.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:05:46 +00:00
/*
* Note that enabling ATS on a VF fails unless it's already enabled
* with the same STU on the PF.
*/
ctrl = PCI_ATS_CTRL_ENABLE;
if (dev->is_virtfn) {
PCI: Use pci_physfn() rather than looking up physfn by hand Use the pci_physfn() helper rather than looking up physfn by hand. No functional change. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:38:13 +00:00
pdev = pci_physfn(dev);
PCI: Embed ATS info directly into struct pci_dev The pci_ats struct is small and will get smaller, so I don't think it's worth allocating it separately from the pci_dev struct. Embed the ATS fields directly into struct pci_dev. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:15:19 +00:00
if (pdev->ats_stu != ps)
PCI: Allocate ATS struct during enumeration Previously, we allocated pci_ats structures when an IOMMU driver called pci_enable_ats(). An SR-IOV VF shares the STU setting with its PF, so when enabling ATS on the VF, we allocated a pci_ats struct for the PF if it didn't already have one. We held the sriov->lock to serialize threads concurrently enabling ATS on several VFS so only one would allocate the PF pci_ats. Gregor reported a deadlock here: pci_enable_sriov sriov_enable virtfn_add mutex_lock(dev->sriov->lock) # acquire sriov->lock pci_device_add device_add BUS_NOTIFY_ADD_DEVICE notifier chain iommu_bus_notifier amd_iommu_add_device # iommu_ops.add_device init_iommu_group iommu_group_get_for_dev iommu_group_add_device __iommu_attach_device amd_iommu_attach_device # iommu_ops.attach_device attach_device pci_enable_ats mutex_lock(dev->sriov->lock) # deadlock There's no reason to delay allocating the pci_ats struct, and if we allocate it for each device at enumeration-time, there's no need for locking in pci_enable_ats(). Allocate pci_ats struct during enumeration, when we initialize other capabilities. Note that this implementation requires ATS to be enabled on the PF first, before on any of the VFs because the PF controls the STU for all the VFs. Link: http://permalink.gmane.org/gmane.linux.kernel.iommu/9433 Reported-by: Gregor Dick <gdick@solarflare.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:05:46 +00:00
return -EINVAL;
} else {
PCI: Embed ATS info directly into struct pci_dev The pci_ats struct is small and will get smaller, so I don't think it's worth allocating it separately from the pci_dev struct. Embed the ATS fields directly into struct pci_dev. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:15:19 +00:00
dev->ats_stu = ps;
ctrl |= PCI_ATS_CTRL_STU(dev->ats_stu - PCI_ATS_MIN_STU);
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
}
PCI: Embed ATS info directly into struct pci_dev The pci_ats struct is small and will get smaller, so I don't think it's worth allocating it separately from the pci_dev struct. Embed the ATS fields directly into struct pci_dev. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:15:19 +00:00
pci_write_config_word(dev, dev->ats_cap + PCI_ATS_CTRL, ctrl);
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
PCI: Embed ATS info directly into struct pci_dev The pci_ats struct is small and will get smaller, so I don't think it's worth allocating it separately from the pci_dev struct. Embed the ATS fields directly into struct pci_dev. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:15:19 +00:00
dev->ats_enabled = 1;
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
return 0;
}
PCI/ATS: Restore EXPORT_SYMBOL_GPL() for pci_{enable,disable}_ats() Commit d355bb209783 ("PCI/ATS: Remove unnecessary EXPORT_SYMBOL_GPL()") unexported a bunch of symbols from the PCI core since the only external users were non-modular IOMMU drivers. Although most of those symbols can remain private for now, 'pci_{enable,disable_ats()' is required for the ARM SMMUv3 driver to build as a module, otherwise we get a build failure as follows: | ERROR: "pci_enable_ats" [drivers/iommu/arm-smmu-v3.ko] undefined! | ERROR: "pci_disable_ats" [drivers/iommu/arm-smmu-v3.ko] undefined! Re-export these two functions so that the ARM SMMUv3 driver can be build as a module. Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@google.com> [will: rewrote commit message] Signed-off-by: Will Deacon <will@kernel.org> Tested-by: John Garry <john.garry@huawei.com> # smmu v3 Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Acked-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-12-19 12:03:39 +00:00
EXPORT_SYMBOL_GPL(pci_enable_ats);
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
/**
* pci_disable_ats - disable the ATS capability
* @dev: the PCI device
*/
void pci_disable_ats(struct pci_dev *dev)
{
u16 ctrl;
PCI: Remove pci_ats_enabled() Remove pci_ats_enabled(). There are no callers outside the ATS code itself. We don't need to check ats_cap, because if we don't find an ATS capability, we'll never set ats_enabled. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-20 14:23:37 +00:00
if (WARN_ON(!dev->ats_enabled))
PCI: Clean up ATS error handling There's no need to BUG() if we enable ATS when it's already enabled. We don't need to BUG() when disabling ATS on a device that doesn't support ATS or if it's already disabled. If ATS is enabled, certainly we found an ATS capability in the past, so it should still be there now. Clean up these error paths. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:43:27 +00:00
return;
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
PCI: Embed ATS info directly into struct pci_dev The pci_ats struct is small and will get smaller, so I don't think it's worth allocating it separately from the pci_dev struct. Embed the ATS fields directly into struct pci_dev. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:15:19 +00:00
pci_read_config_word(dev, dev->ats_cap + PCI_ATS_CTRL, &ctrl);
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
ctrl &= ~PCI_ATS_CTRL_ENABLE;
PCI: Embed ATS info directly into struct pci_dev The pci_ats struct is small and will get smaller, so I don't think it's worth allocating it separately from the pci_dev struct. Embed the ATS fields directly into struct pci_dev. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:15:19 +00:00
pci_write_config_word(dev, dev->ats_cap + PCI_ATS_CTRL, ctrl);
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
PCI: Embed ATS info directly into struct pci_dev The pci_ats struct is small and will get smaller, so I don't think it's worth allocating it separately from the pci_dev struct. Embed the ATS fields directly into struct pci_dev. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:15:19 +00:00
dev->ats_enabled = 0;
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
}
PCI/ATS: Restore EXPORT_SYMBOL_GPL() for pci_{enable,disable}_ats() Commit d355bb209783 ("PCI/ATS: Remove unnecessary EXPORT_SYMBOL_GPL()") unexported a bunch of symbols from the PCI core since the only external users were non-modular IOMMU drivers. Although most of those symbols can remain private for now, 'pci_{enable,disable_ats()' is required for the ARM SMMUv3 driver to build as a module, otherwise we get a build failure as follows: | ERROR: "pci_enable_ats" [drivers/iommu/arm-smmu-v3.ko] undefined! | ERROR: "pci_disable_ats" [drivers/iommu/arm-smmu-v3.ko] undefined! Re-export these two functions so that the ARM SMMUv3 driver can be build as a module. Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <jroedel@suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh@google.com> [will: rewrote commit message] Signed-off-by: Will Deacon <will@kernel.org> Tested-by: John Garry <john.garry@huawei.com> # smmu v3 Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Acked-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-12-19 12:03:39 +00:00
EXPORT_SYMBOL_GPL(pci_disable_ats);
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
PCI: Enable ATS at the device state restore During S3 or S4 resume or PCI reset, ATS regs aren't restored correctly. This patch enables ATS at the device state restore if PCI device has ATS capability. Signed-off-by: Xudong Hao <xudong.hao@intel.com> Signed-off-by: Xiantao Zhang <xiantao.zhang@intel.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-12-17 13:24:40 +00:00
void pci_restore_ats_state(struct pci_dev *dev)
{
u16 ctrl;
PCI: Remove pci_ats_enabled() Remove pci_ats_enabled(). There are no callers outside the ATS code itself. We don't need to check ats_cap, because if we don't find an ATS capability, we'll never set ats_enabled. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-20 14:23:37 +00:00
if (!dev->ats_enabled)
PCI: Enable ATS at the device state restore During S3 or S4 resume or PCI reset, ATS regs aren't restored correctly. This patch enables ATS at the device state restore if PCI device has ATS capability. Signed-off-by: Xudong Hao <xudong.hao@intel.com> Signed-off-by: Xiantao Zhang <xiantao.zhang@intel.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-12-17 13:24:40 +00:00
return;
ctrl = PCI_ATS_CTRL_ENABLE;
if (!dev->is_virtfn)
PCI: Embed ATS info directly into struct pci_dev The pci_ats struct is small and will get smaller, so I don't think it's worth allocating it separately from the pci_dev struct. Embed the ATS fields directly into struct pci_dev. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:15:19 +00:00
ctrl |= PCI_ATS_CTRL_STU(dev->ats_stu - PCI_ATS_MIN_STU);
pci_write_config_word(dev, dev->ats_cap + PCI_ATS_CTRL, ctrl);
PCI: Enable ATS at the device state restore During S3 or S4 resume or PCI reset, ATS regs aren't restored correctly. This patch enables ATS at the device state restore if PCI device has ATS capability. Signed-off-by: Xudong Hao <xudong.hao@intel.com> Signed-off-by: Xiantao Zhang <xiantao.zhang@intel.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-12-17 13:24:40 +00:00
}
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
/**
* pci_ats_queue_depth - query the ATS Invalidate Queue Depth
* @dev: the PCI device
*
* Returns the queue depth on success, or negative on failure.
*
* The ATS spec uses 0 in the Invalidate Queue Depth field to
* indicate that the function can accept 32 Invalidate Request.
* But here we use the `real' values (i.e. 1~32) for the Queue
* Depth; and 0 indicates the function shares the Queue with
* other functions (doesn't exclusively own a Queue).
*/
int pci_ats_queue_depth(struct pci_dev *dev)
{
PCI: Stop caching ATS Invalidate Queue Depth Stop caching the Invalidate Queue Depth in struct pci_dev. pci_ats_queue_depth() is typically called only once per device, and it returns a fixed value per-device, so callers who need the value frequently can cache it themselves. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-20 14:24:32 +00:00
u16 cap;
PCI: Rationalize pci_ats_queue_depth() error checking We previously returned -ENODEV for devices that don't support ATS (except that we always returned 0 for VFs, whether or not they support ATS). For consistency, always return -EINVAL (not -ENODEV) if the device doesn't support ATS. Return zero for VFs that support ATS. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-17 20:30:26 +00:00
if (!dev->ats_cap)
return -EINVAL;
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
if (dev->is_virtfn)
return 0;
PCI: Stop caching ATS Invalidate Queue Depth Stop caching the Invalidate Queue Depth in struct pci_dev. pci_ats_queue_depth() is typically called only once per device, and it returns a fixed value per-device, so callers who need the value frequently can cache it themselves. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2015-07-20 14:24:32 +00:00
pci_read_config_word(dev, dev->ats_cap + PCI_ATS_CAP, &cap);
return PCI_ATS_CAP_QDEP(cap) ? PCI_ATS_CAP_QDEP(cap) : PCI_ATS_MAX_QDEP;
PCI: Move ATS implementation into own file ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capabilities later. Also give ATS its own Kconfig variable to allow selecting it without IOV support. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:13 +00:00
}
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
PCI/ATS: Add pci_ats_page_aligned() interface Return the Page Aligned Request bit in the ATS Capability Register. As per PCIe spec r4.0, sec 10.5.1.2, if the Page Aligned Request bit is set, it indicates the Untranslated Addresses generated by the device are always aligned to a 4096 byte boundary. An IOMMU that can only translate page-aligned addresses can only be used with devices that always produce aligned Untranslated Addresses. This interface will be used by drivers for such IOMMUs to determine whether devices can use the ATS service. Cc: Ashok Raj <ashok.raj@intel.com> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Keith Busch <keith.busch@intel.com> Suggested-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Acked-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2019-02-19 19:06:09 +00:00
/**
* pci_ats_page_aligned - Return Page Aligned Request bit status.
* @pdev: the PCI device
*
* Returns 1, if the Untranslated Addresses generated by the device
* are always aligned or 0 otherwise.
*
* Per PCIe spec r4.0, sec 10.5.1.2, if the Page Aligned Request bit
* is set, it indicates the Untranslated Addresses generated by the
* device are always aligned to a 4096 byte boundary.
*/
int pci_ats_page_aligned(struct pci_dev *pdev)
{
u16 cap;
if (!pdev->ats_cap)
return 0;
pci_read_config_word(pdev, pdev->ats_cap + PCI_ATS_CAP, &cap);
if (cap & PCI_ATS_CAP_PAGE_ALIGNED)
return 1;
return 0;
}
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
#ifdef CONFIG_PCI_PRI
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
void pci_pri_init(struct pci_dev *pdev)
{
PCI/ATS: Cache PRI PRG Response PASID Required bit The PRG Response PASID Required bit in the PRI Capability is read-only. Read it once when we enumerate the device and cache the value so we don't need to read it again. Based-on-patch-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-10-09 21:07:51 +00:00
u16 status;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
pdev->pri_cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PRI);
PCI/ATS: Cache PRI PRG Response PASID Required bit The PRG Response PASID Required bit in the PRI Capability is read-only. Read it once when we enumerate the device and cache the value so we don't need to read it again. Based-on-patch-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-10-09 21:07:51 +00:00
if (!pdev->pri_cap)
return;
pci_read_config_word(pdev, pdev->pri_cap + PCI_PRI_STATUS, &status);
if (status & PCI_PRI_STATUS_PASID)
pdev->pasid_required = 1;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
}
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
/**
* pci_enable_pri - Enable PRI capability
PCI: Fix kerneldoc warnings Fix kerneldoc warnings, e.g., $ make W=1 drivers/pci/ drivers/pci/ats.c:196: warning: Function parameter or member 'pdev' not described in 'pci_enable_pri' drivers/pci/ats.c:196: warning: Function parameter or member 'reqs' not described in 'pci_enable_pri' ... Link: https://lore.kernel.org/r/20200729201224.26799-2-krzk@kernel.org Link: https://lore.kernel.org/r/20200729201224.26799-3-krzk@kernel.org Link: https://lore.kernel.org/r/20200729201224.26799-4-krzk@kernel.org Link: https://lore.kernel.org/r/20200729201224.26799-5-krzk@kernel.org Link: https://lore.kernel.org/r/20200729201224.26799-6-krzk@kernel.org Link: https://lore.kernel.org/r/20200729201224.26799-7-krzk@kernel.org Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2020-07-29 20:12:19 +00:00
* @pdev: PCI device structure
* @reqs: outstanding requests
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
*
* Returns 0 on success, negative value on error
*/
int pci_enable_pri(struct pci_dev *pdev, u32 reqs)
{
u16 control, status;
u32 max_requests;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
int pri = pdev->pri_cap;
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
PCI/ATS: Handle sharing of PF PRI Capability with all VFs Per PCIe r5.0, sec 9.3.7.11, VFs must not implement the PRI Capability. If the PF implements PRI, it is shared by the VFs. Since VFs don't have a PRI Capability, pci_enable_pri() always failed, which caused IOMMU setup to fail. Update the PRI interfaces so for VFs they reflect the state of the PF PRI. [bhelgaas: rebase without pri_cap caching, commit log] Suggested-by: Ashok Raj <ashok.raj@intel.com> Link: https://lore.kernel.org/r/b971e31f8695980da8e4a7f93e3b6a3edba3edaa.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-2-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Ashok Raj <ashok.raj@intel.com> Cc: Keith Busch <keith.busch@intel.com>
2019-09-05 19:31:42 +00:00
/*
* VFs must not implement the PRI Capability. If their PF
* implements PRI, it is shared by the VFs, so if the PF PRI is
* enabled, it is also enabled for the VF.
*/
if (pdev->is_virtfn) {
if (pci_physfn(pdev)->pri_enabled)
return 0;
return -EINVAL;
}
PCI: Cache PRI and PASID bits in pci_dev Device drivers need to check if an IOMMU enabled ATS, PRI and PASID in order to know when they can use the SVM API. Cache PRI and PASID bits in the pci_dev structure, similarly to what is currently done for ATS. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-05-30 16:25:48 +00:00
if (WARN_ON(pdev->pri_enabled))
return -EBUSY;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
if (!pri)
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
return -EINVAL;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
pci_read_config_word(pdev, pri + PCI_PRI_STATUS, &status);
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
if (!(status & PCI_PRI_STATUS_STOPPED))
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
return -EBUSY;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
pci_read_config_dword(pdev, pri + PCI_PRI_MAX_REQ, &max_requests);
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
reqs = min(max_requests, reqs);
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
pdev->pri_reqs_alloc = reqs;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
pci_write_config_dword(pdev, pri + PCI_PRI_ALLOC_REQ, reqs);
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
control = PCI_PRI_CTRL_ENABLE;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
pci_write_config_word(pdev, pri + PCI_PRI_CTRL, control);
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
PCI: Cache PRI and PASID bits in pci_dev Device drivers need to check if an IOMMU enabled ATS, PRI and PASID in order to know when they can use the SVM API. Cache PRI and PASID bits in the pci_dev structure, similarly to what is currently done for ATS. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-05-30 16:25:48 +00:00
pdev->pri_enabled = 1;
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
return 0;
}
/**
* pci_disable_pri - Disable PRI capability
* @pdev: PCI device structure
*
* Only clears the enabled-bit, regardless of its former value
*/
void pci_disable_pri(struct pci_dev *pdev)
{
u16 control;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
int pri = pdev->pri_cap;
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
PCI/ATS: Handle sharing of PF PRI Capability with all VFs Per PCIe r5.0, sec 9.3.7.11, VFs must not implement the PRI Capability. If the PF implements PRI, it is shared by the VFs. Since VFs don't have a PRI Capability, pci_enable_pri() always failed, which caused IOMMU setup to fail. Update the PRI interfaces so for VFs they reflect the state of the PF PRI. [bhelgaas: rebase without pri_cap caching, commit log] Suggested-by: Ashok Raj <ashok.raj@intel.com> Link: https://lore.kernel.org/r/b971e31f8695980da8e4a7f93e3b6a3edba3edaa.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-2-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Ashok Raj <ashok.raj@intel.com> Cc: Keith Busch <keith.busch@intel.com>
2019-09-05 19:31:42 +00:00
/* VFs share the PF PRI */
if (pdev->is_virtfn)
return;
PCI: Cache PRI and PASID bits in pci_dev Device drivers need to check if an IOMMU enabled ATS, PRI and PASID in order to know when they can use the SVM API. Cache PRI and PASID bits in the pci_dev structure, similarly to what is currently done for ATS. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-05-30 16:25:48 +00:00
if (WARN_ON(!pdev->pri_enabled))
return;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
if (!pri)
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
return;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
pci_read_config_word(pdev, pri + PCI_PRI_CTRL, &control);
PCI: More PRI/PASID cleanup More consistency cleanups. Drop the _OFF, separate and indent CTRL/CAP/STATUS bit definitions. This helped find the previous mis-use of bit 0 in the PASID capability register. Reviewed-by: Joerg Roedel <joerg.roedel@amd.com> Tested-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-11-11 17:07:36 +00:00
control &= ~PCI_PRI_CTRL_ENABLE;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
pci_write_config_word(pdev, pri + PCI_PRI_CTRL, control);
PCI: Cache PRI and PASID bits in pci_dev Device drivers need to check if an IOMMU enabled ATS, PRI and PASID in order to know when they can use the SVM API. Cache PRI and PASID bits in the pci_dev structure, similarly to what is currently done for ATS. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-05-30 16:25:48 +00:00
pdev->pri_enabled = 0;
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
}
EXPORT_SYMBOL_GPL(pci_disable_pri);
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
/**
* pci_restore_pri_state - Restore PRI
* @pdev: PCI device structure
*/
void pci_restore_pri_state(struct pci_dev *pdev)
{
u16 control = PCI_PRI_CTRL_ENABLE;
u32 reqs = pdev->pri_reqs_alloc;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
int pri = pdev->pri_cap;
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
PCI/ATS: Handle sharing of PF PRI Capability with all VFs Per PCIe r5.0, sec 9.3.7.11, VFs must not implement the PRI Capability. If the PF implements PRI, it is shared by the VFs. Since VFs don't have a PRI Capability, pci_enable_pri() always failed, which caused IOMMU setup to fail. Update the PRI interfaces so for VFs they reflect the state of the PF PRI. [bhelgaas: rebase without pri_cap caching, commit log] Suggested-by: Ashok Raj <ashok.raj@intel.com> Link: https://lore.kernel.org/r/b971e31f8695980da8e4a7f93e3b6a3edba3edaa.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-2-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Ashok Raj <ashok.raj@intel.com> Cc: Keith Busch <keith.busch@intel.com>
2019-09-05 19:31:42 +00:00
if (pdev->is_virtfn)
return;
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
if (!pdev->pri_enabled)
return;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
if (!pri)
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
return;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
pci_write_config_dword(pdev, pri + PCI_PRI_ALLOC_REQ, reqs);
pci_write_config_word(pdev, pri + PCI_PRI_CTRL, control);
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
}
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
/**
* pci_reset_pri - Resets device's PRI state
* @pdev: PCI device structure
*
* The PRI capability must be disabled before this function is called.
* Returns 0 on success, negative value on error.
*/
int pci_reset_pri(struct pci_dev *pdev)
{
u16 control;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
int pri = pdev->pri_cap;
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
PCI/ATS: Handle sharing of PF PRI Capability with all VFs Per PCIe r5.0, sec 9.3.7.11, VFs must not implement the PRI Capability. If the PF implements PRI, it is shared by the VFs. Since VFs don't have a PRI Capability, pci_enable_pri() always failed, which caused IOMMU setup to fail. Update the PRI interfaces so for VFs they reflect the state of the PF PRI. [bhelgaas: rebase without pri_cap caching, commit log] Suggested-by: Ashok Raj <ashok.raj@intel.com> Link: https://lore.kernel.org/r/b971e31f8695980da8e4a7f93e3b6a3edba3edaa.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-2-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Ashok Raj <ashok.raj@intel.com> Cc: Keith Busch <keith.busch@intel.com>
2019-09-05 19:31:42 +00:00
if (pdev->is_virtfn)
return 0;
PCI: Cache PRI and PASID bits in pci_dev Device drivers need to check if an IOMMU enabled ATS, PRI and PASID in order to know when they can use the SVM API. Cache PRI and PASID bits in the pci_dev structure, similarly to what is currently done for ATS. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-05-30 16:25:48 +00:00
if (WARN_ON(pdev->pri_enabled))
return -EBUSY;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
if (!pri)
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
return -EINVAL;
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
control = PCI_PRI_CTRL_RESET;
PCI/ATS: Cache PRI Capability offset Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pri_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/0c5495d376faf6dbb8eb2165204c474438aaae65.156 7029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-5-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:45 +00:00
pci_write_config_word(pdev, pri + PCI_PRI_CTRL, control);
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
return 0;
}
PCI/ATS: Move pci_prg_resp_pasid_required() to CONFIG_PCI_PRI pci_prg_resp_pasid_required() returns the value of the "PRG Response PASID Required" bit from the PRI capability, but the interface was previously defined under #ifdef CONFIG_PCI_PASID. Move it from CONFIG_PCI_PASID to CONFIG_PCI_PRI so it's with the other PRI-related things. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2019-10-09 19:54:01 +00:00
/**
* pci_prg_resp_pasid_required - Return PRG Response PASID Required bit
* status.
* @pdev: PCI device structure
*
* Returns 1 if PASID is required in PRG Response Message, 0 otherwise.
*/
int pci_prg_resp_pasid_required(struct pci_dev *pdev)
{
PCI/ATS: Handle sharing of PF PRI Capability with all VFs Per PCIe r5.0, sec 9.3.7.11, VFs must not implement the PRI Capability. If the PF implements PRI, it is shared by the VFs. Since VFs don't have a PRI Capability, pci_enable_pri() always failed, which caused IOMMU setup to fail. Update the PRI interfaces so for VFs they reflect the state of the PF PRI. [bhelgaas: rebase without pri_cap caching, commit log] Suggested-by: Ashok Raj <ashok.raj@intel.com> Link: https://lore.kernel.org/r/b971e31f8695980da8e4a7f93e3b6a3edba3edaa.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-2-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Ashok Raj <ashok.raj@intel.com> Cc: Keith Busch <keith.busch@intel.com>
2019-09-05 19:31:42 +00:00
if (pdev->is_virtfn)
pdev = pci_physfn(pdev);
PCI/ATS: Cache PRI PRG Response PASID Required bit The PRG Response PASID Required bit in the PRI Capability is read-only. Read it once when we enumerate the device and cache the value so we don't need to read it again. Based-on-patch-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-10-09 21:07:51 +00:00
return pdev->pasid_required;
PCI/ATS: Move pci_prg_resp_pasid_required() to CONFIG_PCI_PRI pci_prg_resp_pasid_required() returns the value of the "PRG Response PASID Required" bit from the PRI capability, but the interface was previously defined under #ifdef CONFIG_PCI_PASID. Move it from CONFIG_PCI_PASID to CONFIG_PCI_PRI so it's with the other PRI-related things. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Reviewed-by: Joerg Roedel <jroedel@suse.de>
2019-10-09 19:54:01 +00:00
}
PCI/ATS: Add pci_pri_supported() to check device or associated PF For SR-IOV, the PF PRI is shared between the PF and any associated VFs, and the PRI Capability is allowed for PFs but not for VFs. Searching for the PRI Capability on a VF always fails, even if its associated PF supports PRI. Add pci_pri_supported() to check whether device or its associated PF supports PRI. [bhelgaas: commit log, avoid "!!"] Fixes: b16d0cb9e2fc ("iommu/vt-d: Always enable PASID/PRI PCI capabilities before ATS") Link: https://lore.kernel.org/r/1595543849-19692-1-git-send-email-ashok.raj@intel.com Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Acked-by: Joerg Roedel <jroedel@suse.de> Cc: stable@vger.kernel.org # v4.4+
2020-07-23 22:37:29 +00:00
/**
* pci_pri_supported - Check if PRI is supported.
* @pdev: PCI device structure
*
* Returns true if PRI capability is present, false otherwise.
*/
bool pci_pri_supported(struct pci_dev *pdev)
{
/* VFs share the PF PRI */
if (pci_physfn(pdev)->pri_cap)
return true;
return false;
}
EXPORT_SYMBOL_GPL(pci_pri_supported);
PCI: Add implementation for PRI capability Implement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:15 +00:00
#endif /* CONFIG_PCI_PRI */
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
#ifdef CONFIG_PCI_PASID
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
void pci_pasid_init(struct pci_dev *pdev)
{
pdev->pasid_cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_PASID);
}
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
/**
* pci_enable_pasid - Enable the PASID capability
* @pdev: PCI device structure
* @features: Features to enable
*
* Returns 0 on success, negative value on error. This function checks
* whether the features are actually supported by the device and returns
* an error if not.
*/
int pci_enable_pasid(struct pci_dev *pdev, int features)
{
u16 control, supported;
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
int pasid = pdev->pasid_cap;
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
PCI/ATS: Handle sharing of PF PASID Capability with all VFs Per PCIe r5.0, sec 9.3.7.14, if a PF implements the PASID Capability, the PF PASID configuration is shared by its VFs. VFs must not implement their own PASID Capability. Since VFs don't have a PASID Capability, pci_enable_pasid() always failed, which caused IOMMU setup to fail. Update the PASID interfaces so for VFs they reflect the state of the PF PASID. [bhelgaas: rebase without pasid_cap caching, commit log] Suggested-by: Ashok Raj <ashok.raj@intel.com> Link: https://lore.kernel.org/r/8ba1ac192e4ac737508b6ac15002158e176bab91.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-3-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Ashok Raj <ashok.raj@intel.com> Cc: Keith Busch <keith.busch@intel.com>
2019-09-05 19:31:43 +00:00
/*
* VFs must not implement the PASID Capability, but if a PF
* supports PASID, its VFs share the PF PASID configuration.
*/
if (pdev->is_virtfn) {
if (pci_physfn(pdev)->pasid_enabled)
return 0;
return -EINVAL;
}
PCI: Cache PRI and PASID bits in pci_dev Device drivers need to check if an IOMMU enabled ATS, PRI and PASID in order to know when they can use the SVM API. Cache PRI and PASID bits in the pci_dev structure, similarly to what is currently done for ATS. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-05-30 16:25:48 +00:00
if (WARN_ON(pdev->pasid_enabled))
return -EBUSY;
PCI: Allow PASID on fake PCIe devices without TLP prefixes Some systems, e.g., HiSilicon KunPeng920 and KunPeng930, have devices that appear as PCI but are actually on the AMBA bus. Some of these fake PCI devices support a PASID-like feature and they do have a working PASID capability even though they do not use the PCIe Transport Layer Protocol and do not support TLP prefixes. Add a pasid_no_tlp bit for this "PASID works without TLP prefixes" case and update pci_enable_pasid() so it can enable PASID on these devices. Set this bit for HiSilicon KunPeng920 and KunPeng930. [bhelgaas: squashed, commit log] Suggested-by: Bjorn Helgaas <bhelgaas@google.com> Link: https://lore.kernel.org/r/1626144876-11352-2-git-send-email-zhangfei.gao@linaro.org Link: https://lore.kernel.org/r/1626144876-11352-3-git-send-email-zhangfei.gao@linaro.org Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Zhou Wang <wangzhou1@hisilicon.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2021-07-13 02:54:34 +00:00
if (!pdev->eetlp_prefix_path && !pdev->pasid_no_tlp)
PCI: Enable PASID only if entire path supports End-End TLP prefixes A PCIe endpoint carries the process address space identifier (PASID) in the TLP prefix as part of the memory read/write transaction. The address information in the TLP is relevant only for a given PASID context. An IOMMU takes PASID value and the address information from the TLP to look up the physical address in the system. PASID is an End-End TLP Prefix (PCIe r4.0, sec 6.20). Sec 2.2.10.2 says It is an error to receive a TLP with an End-End TLP Prefix by a Receiver that does not support End-End TLP Prefixes. A TLP in violation of this rule is handled as a Malformed TLP. This is a reported error associated with the Receiving Port (see Section 6.2). Prevent error condition by proactively requiring End-End TLP prefix to be supported on the entire data path between the endpoint and the root port before enabling PASID. Signed-off-by: Sinan Kaya <okaya@codeaurora.org> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2018-06-30 15:24:24 +00:00
return -EINVAL;
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
if (!pasid)
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
return -EINVAL;
PCI: Enable PASID only when ACS RR & UF enabled on upstream path The Requester ID/Process Address Space ID (PASID) combination identifies an address space distinct from the PCI bus address space, e.g., an address space defined by an IOMMU. But the PCIe fabric routes Memory Requests based on the TLP address, ignoring any PASID (PCIe r6.0, sec 2.2.10.4), so a TLP with PASID that SHOULD go upstream to the IOMMU may instead be routed as a P2P Request if its address falls in a bridge window. To ensure that all Memory Requests with PASID are routed upstream, only enable PASID if ACS P2P Request Redirect and Upstream Forwarding are enabled for the path leading to the device. Suggested-by: Jason Gunthorpe <jgg@nvidia.com> Suggested-by: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Acked-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Tested-by: Tony Zhu <tony.zhu@intel.com> Link: https://lore.kernel.org/r/20221031005917.45690-5-baolu.lu@linux.intel.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-10-31 00:59:08 +00:00
if (!pci_acs_path_enabled(pdev, NULL, PCI_ACS_RR | PCI_ACS_UF))
return -EINVAL;
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
pci_read_config_word(pdev, pasid + PCI_PASID_CAP, &supported);
PCI: More PRI/PASID cleanup More consistency cleanups. Drop the _OFF, separate and indent CTRL/CAP/STATUS bit definitions. This helped find the previous mis-use of bit 0 in the PASID capability register. Reviewed-by: Joerg Roedel <joerg.roedel@amd.com> Tested-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-11-11 17:07:36 +00:00
supported &= PCI_PASID_CAP_EXEC | PCI_PASID_CAP_PRIV;
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
/* User wants to enable anything unsupported? */
if ((supported & features) != features)
return -EINVAL;
PCI: More PRI/PASID cleanup More consistency cleanups. Drop the _OFF, separate and indent CTRL/CAP/STATUS bit definitions. This helped find the previous mis-use of bit 0 in the PASID capability register. Reviewed-by: Joerg Roedel <joerg.roedel@amd.com> Tested-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-11-11 17:07:36 +00:00
control = PCI_PASID_CTRL_ENABLE | features;
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
pdev->pasid_features = features;
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
pci_write_config_word(pdev, pasid + PCI_PASID_CTRL, control);
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
PCI: Cache PRI and PASID bits in pci_dev Device drivers need to check if an IOMMU enabled ATS, PRI and PASID in order to know when they can use the SVM API. Cache PRI and PASID bits in the pci_dev structure, similarly to what is currently done for ATS. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-05-30 16:25:48 +00:00
pdev->pasid_enabled = 1;
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
return 0;
}
PCI/ATS: Export symbols of PASID functions The Arm SMMUv3 driver uses pci_{enable,disable}_pasid() and related functions. Export them to allow the driver to be built as a module. Acked-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-02-24 16:58:41 +00:00
EXPORT_SYMBOL_GPL(pci_enable_pasid);
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
/**
* pci_disable_pasid - Disable the PASID capability
* @pdev: PCI device structure
*/
void pci_disable_pasid(struct pci_dev *pdev)
{
u16 control = 0;
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
int pasid = pdev->pasid_cap;
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
PCI/ATS: Handle sharing of PF PASID Capability with all VFs Per PCIe r5.0, sec 9.3.7.14, if a PF implements the PASID Capability, the PF PASID configuration is shared by its VFs. VFs must not implement their own PASID Capability. Since VFs don't have a PASID Capability, pci_enable_pasid() always failed, which caused IOMMU setup to fail. Update the PASID interfaces so for VFs they reflect the state of the PF PASID. [bhelgaas: rebase without pasid_cap caching, commit log] Suggested-by: Ashok Raj <ashok.raj@intel.com> Link: https://lore.kernel.org/r/8ba1ac192e4ac737508b6ac15002158e176bab91.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-3-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Ashok Raj <ashok.raj@intel.com> Cc: Keith Busch <keith.busch@intel.com>
2019-09-05 19:31:43 +00:00
/* VFs share the PF PASID configuration */
if (pdev->is_virtfn)
return;
PCI: Cache PRI and PASID bits in pci_dev Device drivers need to check if an IOMMU enabled ATS, PRI and PASID in order to know when they can use the SVM API. Cache PRI and PASID bits in the pci_dev structure, similarly to what is currently done for ATS. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-05-30 16:25:48 +00:00
if (WARN_ON(!pdev->pasid_enabled))
return;
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
if (!pasid)
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
return;
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
pci_write_config_word(pdev, pasid + PCI_PASID_CTRL, control);
PCI: Cache PRI and PASID bits in pci_dev Device drivers need to check if an IOMMU enabled ATS, PRI and PASID in order to know when they can use the SVM API. Cache PRI and PASID bits in the pci_dev structure, similarly to what is currently done for ATS. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-05-30 16:25:48 +00:00
pdev->pasid_enabled = 0;
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
}
PCI/ATS: Export symbols of PASID functions The Arm SMMUv3 driver uses pci_{enable,disable}_pasid() and related functions. Export them to allow the driver to be built as a module. Acked-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-02-24 16:58:41 +00:00
EXPORT_SYMBOL_GPL(pci_disable_pasid);
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
/**
* pci_restore_pasid_state - Restore PASID capabilities
* @pdev: PCI device structure
*/
void pci_restore_pasid_state(struct pci_dev *pdev)
{
u16 control;
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
int pasid = pdev->pasid_cap;
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
PCI/ATS: Handle sharing of PF PASID Capability with all VFs Per PCIe r5.0, sec 9.3.7.14, if a PF implements the PASID Capability, the PF PASID configuration is shared by its VFs. VFs must not implement their own PASID Capability. Since VFs don't have a PASID Capability, pci_enable_pasid() always failed, which caused IOMMU setup to fail. Update the PASID interfaces so for VFs they reflect the state of the PF PASID. [bhelgaas: rebase without pasid_cap caching, commit log] Suggested-by: Ashok Raj <ashok.raj@intel.com> Link: https://lore.kernel.org/r/8ba1ac192e4ac737508b6ac15002158e176bab91.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-3-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Ashok Raj <ashok.raj@intel.com> Cc: Keith Busch <keith.busch@intel.com>
2019-09-05 19:31:43 +00:00
if (pdev->is_virtfn)
return;
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
if (!pdev->pasid_enabled)
return;
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
if (!pasid)
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
return;
control = PCI_PASID_CTRL_ENABLE | pdev->pasid_features;
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
pci_write_config_word(pdev, pasid + PCI_PASID_CTRL, control);
PCI: Restore PRI and PASID state after Function-Level Reset After a Function-Level Reset, PCI states need to be restored. Save PASID features and PRI reqs cached. [bhelgaas: search for capability only if PRI/PASID were enabled] Signed-off-by: CQ Tang <cq.tang@intel.com> Signed-off-by: Ashok Raj <ashok.raj@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Jean-Phillipe Brucker <jean-philippe.brucker@arm.com> Cc: David Woodhouse <dwmw2@infradead.org>
2017-05-30 16:25:49 +00:00
}
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
/**
* pci_pasid_features - Check which PASID features are supported
* @pdev: PCI device structure
*
* Returns a negative value when no PASI capability is present.
* Otherwise is returns a bitmask with supported features. Current
* features reported are:
PCI: More PRI/PASID cleanup More consistency cleanups. Drop the _OFF, separate and indent CTRL/CAP/STATUS bit definitions. This helped find the previous mis-use of bit 0 in the PASID capability register. Reviewed-by: Joerg Roedel <joerg.roedel@amd.com> Tested-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-11-11 17:07:36 +00:00
* PCI_PASID_CAP_EXEC - Execute permission supported
PCI: Fix whitespace, capitalization, and spelling errors Fix whitespace, capitalization, and spelling errors. No functional change. I know "busses" is not an error, but "buses" was more common, so I used it consistently. Signed-off-by: Marta Rybczynska <rybczynska@gmail.com> (pci_reset_bridge_secondary_bus()) Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2013-11-14 18:28:18 +00:00
* PCI_PASID_CAP_PRIV - Privileged mode supported
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
*/
int pci_pasid_features(struct pci_dev *pdev)
{
u16 supported;
PCI/ATS: Use PF PASID for VFs Per PCIe r5.0, sec 9.3.7.14, if a PF implements the PASID Capability, the PF PASID configuration is shared by its VFs, and VFs must not implement their own PASID Capability. But commit 751035b8dc06 ("PCI/ATS: Cache PASID Capability offset") changed pci_max_pasids() and pci_pasid_features() to use the PASID Capability of the VF device instead of the associated PF device. This leads to IOMMU bind failures when pci_max_pasids() and pci_pasid_features() are called for VFs. In pci_max_pasids() and pci_pasid_features(), always use the PF PASID Capability. Fixes: 751035b8dc06 ("PCI/ATS: Cache PASID Capability offset") Link: https://lore.kernel.org/r/fe891f9755cb18349389609e7fed9940fc5b081a.1580325170.git.sathyanarayanan.kuppuswamy@linux.intel.com Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> CC: stable@vger.kernel.org # v5.5+
2020-01-29 19:14:00 +00:00
int pasid;
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
PCI/ATS: Handle sharing of PF PASID Capability with all VFs Per PCIe r5.0, sec 9.3.7.14, if a PF implements the PASID Capability, the PF PASID configuration is shared by its VFs. VFs must not implement their own PASID Capability. Since VFs don't have a PASID Capability, pci_enable_pasid() always failed, which caused IOMMU setup to fail. Update the PASID interfaces so for VFs they reflect the state of the PF PASID. [bhelgaas: rebase without pasid_cap caching, commit log] Suggested-by: Ashok Raj <ashok.raj@intel.com> Link: https://lore.kernel.org/r/8ba1ac192e4ac737508b6ac15002158e176bab91.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-3-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Ashok Raj <ashok.raj@intel.com> Cc: Keith Busch <keith.busch@intel.com>
2019-09-05 19:31:43 +00:00
if (pdev->is_virtfn)
pdev = pci_physfn(pdev);
PCI/ATS: Use PF PASID for VFs Per PCIe r5.0, sec 9.3.7.14, if a PF implements the PASID Capability, the PF PASID configuration is shared by its VFs, and VFs must not implement their own PASID Capability. But commit 751035b8dc06 ("PCI/ATS: Cache PASID Capability offset") changed pci_max_pasids() and pci_pasid_features() to use the PASID Capability of the VF device instead of the associated PF device. This leads to IOMMU bind failures when pci_max_pasids() and pci_pasid_features() are called for VFs. In pci_max_pasids() and pci_pasid_features(), always use the PF PASID Capability. Fixes: 751035b8dc06 ("PCI/ATS: Cache PASID Capability offset") Link: https://lore.kernel.org/r/fe891f9755cb18349389609e7fed9940fc5b081a.1580325170.git.sathyanarayanan.kuppuswamy@linux.intel.com Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> CC: stable@vger.kernel.org # v5.5+
2020-01-29 19:14:00 +00:00
pasid = pdev->pasid_cap;
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
if (!pasid)
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
return -EINVAL;
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
pci_read_config_word(pdev, pasid + PCI_PASID_CAP, &supported);
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
PCI: More PRI/PASID cleanup More consistency cleanups. Drop the _OFF, separate and indent CTRL/CAP/STATUS bit definitions. This helped find the previous mis-use of bit 0 in the PASID capability register. Reviewed-by: Joerg Roedel <joerg.roedel@amd.com> Tested-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-11-11 17:07:36 +00:00
supported &= PCI_PASID_CAP_EXEC | PCI_PASID_CAP_PRIV;
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
return supported;
}
PCI/ATS: Export symbols of PASID functions The Arm SMMUv3 driver uses pci_{enable,disable}_pasid() and related functions. Export them to allow the driver to be built as a module. Acked-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-02-24 16:58:41 +00:00
EXPORT_SYMBOL_GPL(pci_pasid_features);
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
/**
PCI: Fix kernel-doc errors Fix kernel-doc formatting errors, function names that don't match the doc, and some missing parameter documentation. These are reported by: make W=1 drivers/pci/ No functional change intended. [bhelgaas: squashed into one patch since this only changes comments] Link: https://lore.kernel.org/r/20210311001724.423356-1-kw@linux.com Link: https://lore.kernel.org/r/20210311001724.423356-2-kw@linux.com Link: https://lore.kernel.org/r/20210311001724.423356-3-kw@linux.com Link: https://lore.kernel.org/r/20210311001724.423356-4-kw@linux.com Link: https://lore.kernel.org/r/20210311001724.423356-5-kw@linux.com Link: https://lore.kernel.org/r/20210311001724.423356-6-kw@linux.com Link: https://lore.kernel.org/r/20210311001724.423356-7-kw@linux.com Link: https://lore.kernel.org/r/20210311001724.423356-8-kw@linux.com Signed-off-by: Krzysztof Wilczyński <kw@linux.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2021-03-11 00:17:17 +00:00
* pci_max_pasids - Get maximum number of PASIDs supported by device
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
* @pdev: PCI device structure
*
* Returns negative value when PASID capability is not present.
PCI: Fix typos and whitespace errors Fix typos in drivers/pci. Comment and whitespace changes only. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
2019-05-30 13:05:58 +00:00
* Otherwise it returns the number of supported PASIDs.
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
*/
int pci_max_pasids(struct pci_dev *pdev)
{
u16 supported;
PCI/ATS: Use PF PASID for VFs Per PCIe r5.0, sec 9.3.7.14, if a PF implements the PASID Capability, the PF PASID configuration is shared by its VFs, and VFs must not implement their own PASID Capability. But commit 751035b8dc06 ("PCI/ATS: Cache PASID Capability offset") changed pci_max_pasids() and pci_pasid_features() to use the PASID Capability of the VF device instead of the associated PF device. This leads to IOMMU bind failures when pci_max_pasids() and pci_pasid_features() are called for VFs. In pci_max_pasids() and pci_pasid_features(), always use the PF PASID Capability. Fixes: 751035b8dc06 ("PCI/ATS: Cache PASID Capability offset") Link: https://lore.kernel.org/r/fe891f9755cb18349389609e7fed9940fc5b081a.1580325170.git.sathyanarayanan.kuppuswamy@linux.intel.com Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> CC: stable@vger.kernel.org # v5.5+
2020-01-29 19:14:00 +00:00
int pasid;
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
PCI/ATS: Handle sharing of PF PASID Capability with all VFs Per PCIe r5.0, sec 9.3.7.14, if a PF implements the PASID Capability, the PF PASID configuration is shared by its VFs. VFs must not implement their own PASID Capability. Since VFs don't have a PASID Capability, pci_enable_pasid() always failed, which caused IOMMU setup to fail. Update the PASID interfaces so for VFs they reflect the state of the PF PASID. [bhelgaas: rebase without pasid_cap caching, commit log] Suggested-by: Ashok Raj <ashok.raj@intel.com> Link: https://lore.kernel.org/r/8ba1ac192e4ac737508b6ac15002158e176bab91.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-3-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Ashok Raj <ashok.raj@intel.com> Cc: Keith Busch <keith.busch@intel.com>
2019-09-05 19:31:43 +00:00
if (pdev->is_virtfn)
pdev = pci_physfn(pdev);
PCI/ATS: Use PF PASID for VFs Per PCIe r5.0, sec 9.3.7.14, if a PF implements the PASID Capability, the PF PASID configuration is shared by its VFs, and VFs must not implement their own PASID Capability. But commit 751035b8dc06 ("PCI/ATS: Cache PASID Capability offset") changed pci_max_pasids() and pci_pasid_features() to use the PASID Capability of the VF device instead of the associated PF device. This leads to IOMMU bind failures when pci_max_pasids() and pci_pasid_features() are called for VFs. In pci_max_pasids() and pci_pasid_features(), always use the PF PASID Capability. Fixes: 751035b8dc06 ("PCI/ATS: Cache PASID Capability offset") Link: https://lore.kernel.org/r/fe891f9755cb18349389609e7fed9940fc5b081a.1580325170.git.sathyanarayanan.kuppuswamy@linux.intel.com Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> CC: stable@vger.kernel.org # v5.5+
2020-01-29 19:14:00 +00:00
pasid = pdev->pasid_cap;
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
if (!pasid)
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
return -EINVAL;
PCI/ATS: Cache PASID Capability offset Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time. [bhelgaas: commit log, reorder patch to later, call pci_pasid_init() from pci_init_capabilities()] Link: https://lore.kernel.org/r/4957778959fa34eab3e8b3065d1951989c61cb0f.1567029860.git.sathyanarayanan.kuppuswamy@linux.intel.com Link: https://lore.kernel.org/r/20190905193146.90250-6-helgaas@kernel.org Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-09-05 19:31:46 +00:00
pci_read_config_word(pdev, pasid + PCI_PASID_CAP, &supported);
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
PCI/ATS: Use FIELD_GET() Use FIELD_GET() to remove dependences on the field position, i.e., the shift value. No functional change intended. Link: https://lore.kernel.org/r/20231010204436.1000644-6-helgaas@kernel.org Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com> Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
2023-10-10 20:44:31 +00:00
return (1 << FIELD_GET(PCI_PASID_CAP_WIDTH, supported));
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
}
PCI/ATS: Export symbols of PASID functions The Arm SMMUv3 driver uses pci_{enable,disable}_pasid() and related functions. Export them to allow the driver to be built as a module. Acked-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Signed-off-by: Will Deacon <will@kernel.org>
2020-02-24 16:58:41 +00:00
EXPORT_SYMBOL_GPL(pci_max_pasids);
PCI: Add support for PASID capability Devices supporting Process Address Space Identifiers (PASIDs) can use an IOMMU to access multiple IO address spaces at the same time. A PCIe device indicates support for this feature by implementing the PASID capability. This patch adds support for the capability to the Linux kernel. Reviewed-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2011-09-27 13:57:16 +00:00
#endif /* CONFIG_PCI_PASID */
Reference in New Issue Copy Permalink