- Cleanup pciehp LED/indicator control with a new consolidated
    pciehp_set_indicators() interface that controls both Attention and
    Power Indicators (Denis Efremov)

* pci/pciehp:
  PCI: pciehp: Refer to "Indicators" instead of "LEDs" in comments
  PCI: pciehp: Remove pciehp_green_led_{on,off,blink}()
  PCI: pciehp: Remove pciehp_set_attention_status()
  PCI: pciehp: Combine adjacent indicator updates
  PCI: pciehp: Add pciehp_set_indicators() to set both indicators

  - Move P2PCMA PCI bus offset from generic dev_pagemap to
    pci_p2pdma_pagemap (Logan Gunthorpe)

  - Add provider's pci_dev to pci_p2pdma_pagemap (Logan Gunthorpe)

  - Apply host bridge whitelist for ACS (Logan Gunthorpe)

  - Whitelist some Intel host bridges for P2PDMA (Logan Gunthorpe)

  - Add attrs to pci_p2pdma_map_sg() to match dma_map_sg() (Logan
    Gunthorpe)

  - Add pci_p2pdma_unmap_sg() (Logan Gunthorpe)

  - Store P2PDMA mapping method in xarray (Logan Gunthorpe)

  - Map requests that traverse a host bridge (Logan Gunthorpe)

  - Allow IOMMU for host bridge whitelist (Logan Gunthorpe)

* pci/p2pdma:
  PCI/P2PDMA: Update pci_p2pdma_distance_many() documentation
  PCI/P2PDMA: Allow IOMMU for host bridge whitelist
  PCI/P2PDMA: dma_map() requests that traverse the host bridge
  PCI/P2PDMA: Store mapping method in an xarray
  PCI/P2PDMA: Factor out __pci_p2pdma_map_sg()
  PCI/P2PDMA: Introduce pci_p2pdma_unmap_sg()
  PCI/P2PDMA: Add attrs argument to pci_p2pdma_map_sg()
  PCI/P2PDMA: Whitelist some Intel host bridges
  PCI/P2PDMA: Factor out host_bridge_whitelist()
  PCI/P2PDMA: Apply host bridge whitelist for ACS
  PCI/P2PDMA: Factor out __upstream_bridge_distance()
  PCI/P2PDMA: Add constants for map type results to upstream_bridge_distance()
  PCI/P2PDMA: Add provider's pci_dev to pci_p2pdma_pagemap struct
  PCI/P2PDMA: Introduce private pagemap structure

  - Enable PCI_MSI_IRQ_DOMAIN support for RISC-V (Wesley Terpstra)

* pci/msi:
  PCI/MSI: Enable PCI_MSI_IRQ_DOMAIN support for RISC-V

  - Use devm_add_action_or_reset() helper (Fuqian Huang)

  - Mark expected switch fall-through (Gustavo A. R. Silva)

  - Convert sysfs device attributes from __ATTR() to DEVICE_ATTR() (Kelsey
    Skunberg)

  - Convert sysfs file permissions from S_IRUSR etc to octal (Kelsey
    Skunberg)

  - Move SR-IOV sysfs functions to iov.c (Kelsey Skunberg)

  - Add pci_info_ratelimited() to ratelimit PCI messages separately
    (Krzysztof Wilczynski)

  - Fix "'static' not at beginning of declaration" warnings (Krzysztof
    Wilczynski)

  - Clean up resource_alignment parameter to not require static buffer
    (Logan Gunthorpe)

  - Add ACS quirk for iProc PAXB (Abhinav Ratna)

  - Add pci_irq_vector() and other stubs for !CONFIG_PCI (Herbert Xu)

* pci/misc:
  PCI: Add pci_irq_vector() and other stubs when !CONFIG_PCI
  PCI: Add ACS quirk for iProc PAXB
  PCI: Force trailing new line to resource_alignment_param in sysfs
  PCI: Move pci_[get|set]_resource_alignment_param() into their callers
  PCI: Clean up resource_alignment parameter to not require static buffer
  PCI: Use static const struct, not const static struct
  PCI: Add pci_info_ratelimited() to ratelimit PCI separately
  PCI/IOV: Remove group write permission from sriov_numvfs, sriov_drivers_autoprobe
  PCI/IOV: Move sysfs SR-IOV functions to iov.c
  PCI: sysfs: Change permissions from symbolic to octal
  PCI: sysfs: Change DEVICE_ATTR() to DEVICE_ATTR_WO()
  PCI: sysfs: Define device attributes with DEVICE_ATTR*()
  PCI: Mark expected switch fall-through
  PCI: Use devm_add_action_or_reset()

  - Consolidate _HPP & _HPX code in pci-acpi.h and remove unnecessary
    struct hotplug_program_ops (Krzysztof Wilczynski)

  - Fixup PCIe device types to remove the need for dev->has_secondary_link
    (Mika Westerberg)

* pci/enumeration:
  PCI: Get rid of dev->has_secondary_link flag
  PCI: Make pcie_downstream_port() available outside of access.c
  PCI/ACPI: Remove unnecessary struct hotplug_program_ops
  PCI/ACPI: Move _HPP & _HPX functions to pci-acpi.c
  PCI/ACPI: Rename _HPX structs from hpp_* to hpx_*

  - Move many symbols from public linux/pci.h to subsystem-private
    drivers/pci/pci.h (Kelsey Skunberg)

  - Unexport pci_bus_get() and pci_bus_sem since they're not needed by
    modules (Kelsey Skunberg)

  - Remove unused pci_block_cfg_access() et al (Kelsey Skunberg)

* pci/encapsulate:
  PCI: Make pci_set_of_node(), etc private
  PCI: Make pci_enable_ptm() private
  PCI: Make pcie_set_ecrc_checking(), pcie_ecrc_get_policy() private
  PCI: Make pci_ats_init() private
  PCI: Make pcie_update_link_speed() private
  PCI: Make pci_bus_get(), pci_bus_put() private
  PCI: Make pci_hotplug_io_size, mem_size, and bus_size private
  PCI: Make pci_save_vc_state(), pci_restore_vc_state(), etc private
  PCI: Make pci_get_host_bridge_device(), pci_put_host_bridge_device() private
  PCI: Make pci_check_pme_status(), pci_pme_wakeup_bus() private
  PCI: Make PCI_PM_* delay times private
  PCI: Unexport pci_bus_sem
  PCI: Unexport pci_bus_get() and pci_bus_put()
  PCI: Remove pci_block_cfg_access() et al (unused)

  - Consolidate ASPM interfaces in linux/pci.h (Krzysztof Wilczynski)

* pci/aspm:
  PCI: Move ASPM declarations to linux/pci.h

Add stub functions pci_alloc_irq_vectors_affinity() and pci_irq_vector()
when CONFIG_PCI is off so drivers can use them without resorting to ifdefs.

Also move the PCI_IRQ_* macros outside of the ifdefs so they are always
available.

Fixes: 625f269a ("crypto: inside-secure - add support for PCI based FPGA development board")
Link: https://lore.kernel.org/r/20190904122600.GA28660@gondor.apana.org.auReported-by: kbuild test robot <lkp@intel.com>
Reported-by: YueHaibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

In some systems, the Device/Port Type in the PCI Express Capabilities
register incorrectly identifies upstream ports as downstream ports.

d0751b98 ("PCI: Add dev->has_secondary_link to track downstream PCIe
links") addressed this by adding pci_dev.has_secondary_link, which is set
for downstream ports.  But this is confusing because pci_pcie_type()
sometimes gives the wrong answer, and it's not obvious that we should use
pci_dev.has_secondary_link instead.

Reduce the confusion by correcting the type of the port itself so that
pci_pcie_type() returns the actual type regardless of what the Device/Port
Type register claims it is.  Update the users to call pci_pcie_type() and
pcie_downstream_port() accordingly, and remove pci_dev.has_secondary_link
completely.

Link: https://lore.kernel.org/linux-pci/20190703133953.GK128603@google.com/Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20190822085553.62697-2-mika.westerberg@linux.intel.comSigned-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

pcie_downstream_port() is useful in other places where code needs to
determine whether the PCIe port is downstream so make it available outside
of access.c.

Link: https://lore.kernel.org/r/20190822085553.62697-1-mika.westerberg@linux.intel.comSigned-off-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

iProc PAXB Root Ports don't advertise an ACS capability, but they do not
allow peer-to-peer transactions between Root Ports.  Add an ACS quirk so
each Root Port can be in a separate IOMMU group.

[bhelgaas: commit log, comment, use common implementation style]
Link: https://lore.kernel.org/r/1566275985-25670-1-git-send-email-srinath.mannam@broadcom.comSigned-off-by: Abhinav Ratna <abhinav.ratna@broadcom.com>
Signed-off-by: Srinath Mannam <srinath.mannam@broadcom.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Scott Branden <scott.branden@broadcom.com>

When 'pci=resource_alignment=' is specified on the command line, there is
no trailing new line.  Then, when it's read through the corresponding sysfs
attribute, there will be no newline and a cat command will not show
correctly in a shell. If the parameter is set through sysfs a new line will
be stored and it will 'cat' correctly.

To solve this, append a new line character in the show function if one does
not already exist.

Link: https://lore.kernel.org/r/20190822161013.5481-4-logang@deltatee.comSigned-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Both the functions pci_get_resource_alignment_param() and
pci_set_resource_alignment_param() are now only called in one place:
resource_alignment_show() and resource_alignment_store() respectively.

There is no value in this extra set of functions so move both into their
callers respectively.

[bhelgaas: fold in "GFP_KERNEL while atomic" fix from Christoph Hellwig
<hch@infradead.org>
https://lore.kernel.org/r/20190902075006.GB754@infradead.org]
Link: https://lore.kernel.org/r/20190822161013.5481-3-logang@deltatee.comSigned-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

The PCIe spec doesn't mention "green LEDs" or "amber LEDs".  Replace those
terms with "Power Indicator" and "Attention Indicator" so the comments
match the spec language.  Comment changes only.
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Remove pciehp_green_led_{on,off,blink}() and use pciehp_set_indicators()
instead, since the code is mostly the same.

[bhelgaas: drop set_power_indicator() wrapper to reduce the number of
interfaces]
Link: https://lore.kernel.org/r/20190903111021.1559-5-efremov@linux.comSigned-off-by: Denis Efremov <efremov@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

Remove pciehp_set_attention_status() and use pciehp_set_indicators()
instead, since the code is mostly the same.

Link: https://lore.kernel.org/r/20190903111021.1559-4-efremov@linux.comSigned-off-by: Denis Efremov <efremov@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

Combine adjacent updates of power and attention indicators into a single
pciehp_set_indicators() call.  This sends one command to the hotplug
controller instead of two.

Link: https://lore.kernel.org/r/20190903111021.1559-3-efremov@linux.comSigned-off-by: Denis Efremov <efremov@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Reviewed-by: Lukas Wunner <lukas@wunner.de>

Add pciehp_set_indicators() to set power and attention indicators with a
single register write.

This is a minor optimization because we frequently set both indicators and
this can do it with a single command.  It also reduces the number of
interfaces related to the indicators and makes them more discoverable
because callers use the PCI_EXP_SLTCTL_ATTN_IND_* and
PCI_EXP_SLTCTL_PWR_IND_* definitions directly.

[bhelgaas: extend commit log, s/PCI_EXP_SLTCTL_.*_IND_NONE/INDICATOR_NOOP/
so they don't look like things defined by the spec, add function doc, mask
commands to make it obvious we only send valid commands
(pcie_do_write_cmd() does mask it, but requires more effort to verify)]
Link: https://lore.kernel.org/r/20190903111021.1559-2-efremov@linux.comSigned-off-by: Denis Efremov <efremov@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Clean up the 'resource_alignment' parameter code to use kstrdup() in the
initcall routine instead of a static buffer that wastes memory regardless
of whether the feature is used.  This allows us to drop 'COMMAND_LINE_SIZE'
bytes (typically 256-4096 depending on architecture) of static data.

This is similar to what has been done for the 'disable_acs_redir'
parameter.

Link: https://lore.kernel.org/r/20190822161013.5481-2-logang@deltatee.comSigned-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Move the static keyword to the front of declarations of pci_regs_behavior[]
and pcie_cap_regs_behavior[], which resolves compiler warnings when
building with "W=1":

  drivers/pci/pci-bridge-emul.c:41:1: warning: ‘static’ is not at beginning of
  declaration [-Wold-style-declaration]
   const static struct pci_bridge_reg_behavior pci_regs_behavior[] = {
   ^
  drivers/pci/pci-bridge-emul.c:176:1: warning: ‘static’ is not at beginning of
  declaration [-Wold-style-declaration]
   const static struct pci_bridge_reg_behavior pcie_cap_regs_behavior[] = {
   ^

Link: https://lore.kernel.org/r/20190826151436.4672-1-kw@linux.com
Link: https://lore.kernel.org/r/20190828131733.5817-1-kw@linux.comSigned-off-by: Krzysztof Wilczynski <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>

Do not use printk_ratelimit() in drivers/pci/pci.c as it shares the rate
limiting state with all other callers to the printk_ratelimit().

Add pci_info_ratelimited() (similar to pci_notice_ratelimited() added in
the commit a88a7b3e ("vfio: Use dev_printk() when possible")) and use
it instead of printk_ratelimit() + pci_info().

Link: https://lore.kernel.org/r/20190825224616.8021-1-kw@linux.comSigned-off-by: Krzysztof Wilczynski <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Previously the sriov_numvfs and sriov_drivers_autoprobe sysfs files had
0664 permissions, which allowed group write.  libvirt runs as root when
dealing with PCI, and it chowns files needed by qemu, so group write
permission should not be needed.

Change these permissions from 0664 to 0644, which is what DEVICE_ATTR_RW()
does by default.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20190905063226.43269-1-skunberg.kelsey@gmail.com
Link: https://lore.kernel.org/r/850cf536-0b72-d78c-efaf-855dcb391087@redhat.comSigned-off-by: Kelsey Skunberg <skunberg.kelsey@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Donald Dutile <ddutile@redhat.com>

Move the ACPI-specific structs hpx_type0, hpx_type1, hpx_type2 and
hpx_type3 to drivers/pci/pci-acpi.c as they are not used anywhere else.
Then remove the struct hotplug_program_ops that has been shared between
drivers/pci/probe.c and drivers/pci/pci-acpi.c from drivers/pci/pci.h as it
is no longer needed.

The struct hotplug_program_ops was added by 87fcf12e ("PCI/ACPI: Remove
the need for 'struct hotplug_params'") and replaced previously used struct
hotplug_params enabling the support for the _HPX Type 3 Setting Record that
was added by f873c51a ("PCI/ACPI: Implement _HPX Type 3 Setting
Record").

The new struct allowed for the static functions such program_hpx_type0(),
program_hpx_type1(), etc., from the drivers/pci/probe.c to be called from
the function pci_acpi_program_hp_params() in the drivers/pci/pci-acpi.c.

Previously a programming of _HPX Type 0 was as follows:

  drivers/pci/probe.c:

    program_hpx_type0()
    ...
    pci_configure_device()
      hp_ops = {
        .program_type0 = program_hpx_type0,
        ...
      }
      pci_acpi_program_hp_params(&hp_ops)

  drivers/pci/pci-acpi.c:

    pci_acpi_program_hp_params(&hp_ops)
      acpi_run_hpx(hp_ops)
        decode_type0_hpx_record()
          hp_ops->program_type0     # program_hpx_type0() called via hp_ops

After the ACPI-specific functions, structs, enums, etc., have been moved to
drivers/pci/pci-acpi.c there is no need for the hotplug_program_ops as all
of the _HPX Type 0, 1, 2 and 3 are directly accessible.

Link: https://lore.kernel.org/r/20190827094951.10613-4-kw@linux.comSigned-off-by: Krzysztof Wilczynski <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Move program_hpx_type0(), program_hpx_type1(), etc., and enums
hpx_type3_dev_type, hpx_type3_fn_type and hpx_type3_cfg_loc to
drivers/pci/pci-acpi.c as these functions and enums are ACPI-specific.

Move structs hpx_type0, hpx_type1, hpx_type2 and hpx_type3 to
drivers/pci/pci.h as these are shared between drivers/pci/pci-acpi.c and
drivers/pci/probe.c.

Link: https://lore.kernel.org/r/20190827094951.10613-3-kw@linux.comSigned-off-by: Krzysztof Wilczynski <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

The names of the hpp_type0, hpp_type1 and hpp_type2 structs suggest that
they're related to _HPP, when in fact they're related to _HPX.

The struct hpp_type0 denotes an _HPX Type 0 setting record that supersedes
the _HPP setting record, and it has been used interchangeably for _HPP as
per the ACPI specification (see version 6.3, section 6.2.9.1) which states
that it should be applied to PCI, PCI-X and PCI Express devices, with
settings being ignored if they are not applicable.

Rename them to hpx_type0, hpx_type1 and hpx_type2 to reflect their relation
to _HPX rather than _HPP.

Link: https://lore.kernel.org/r/20190827094951.10613-2-kw@linux.comSigned-off-by: Krzysztof Wilczynski <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Move ASPM definitions and function prototypes from include/linux/pci-aspm.h
to include/linux/pci.h so users only need to include <linux/pci.h>:

  PCIE_LINK_STATE_L0S
  PCIE_LINK_STATE_L1
  PCIE_LINK_STATE_CLKPM
  pci_disable_link_state()
  pci_disable_link_state_locked()
  pcie_no_aspm()

No functional changes intended.

Link: https://lore.kernel.org/r/20190827095620.11213-1-kw@linux.comSigned-off-by: Krzysztof Wilczynski <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

The sysfs SR-IOV functions are only needed when the kernel is built with
SR-IOV support.  Rather than put them in pci-sysfs.c under #ifdef
CONFIG_PCI_IOV, move them to iov.c, which is only compiled when
CONFIG_PCI_IOV=y.

Link: https://lore.kernel.org/r/20190813204513.4790-4-skunberg.kelsey@gmail.comSigned-off-by: Kelsey Skunberg <skunberg.kelsey@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Donald Dutile <ddutile@redhat.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

We prefer octal permissions over symbolic permissions such as "(S_IWUSR |
S_IWGRP)".  Change all symbolic permissions to octal permissions, e.g.,

  - (S_IWUSR | S_IWGRP)
  + 0220

Link: https://lore.kernel.org/r/20190813204513.4790-3-skunberg.kelsey@gmail.comSigned-off-by: Kelsey Skunberg <skunberg.kelsey@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Donald Dutile <ddutile@redhat.com>

DEVICE_ATTR() should only be used when files have unusual permissions.
Change DEVICE_ATTR() with '0220' write-only permissions to
DEVICE_ATTR_WO(), e.g.,

  - static DEVICE_ATTR(_name, (S_IWUSR | S_IWGRP), NULL, _store);
  + static DEVICE_ATTR_WO(_name);

Since _store is no longer passed, make the _name passed by DEVICE_ATTR_WO()
and the related _name##_store() name match with each other, e.g.,

  DEVICE_ATTR_WO(bus_rescan) must be able to call bus_rescan_store()

Link: https://lore.kernel.org/r/20190815153352.86143-4-skunberg.kelsey@gmail.comSigned-off-by: Kelsey Skunberg <skunberg.kelsey@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Donald Dutile <ddutile@redhat.com>

Device attributes should be defined using DEVICE_ATTR*(_name, _mode, _show,
_store).  Convert them all from __ATTR*() to DEVICE_ATTR*(), e.g.,

  - struct device_attribute dev_attr_##_name = __ATTR(_name, _mode, _show,
                                                      _store)
  + static DEVICE_ATTR(foo, S_IWUSR | S_IRUGO, show_foo, store_foo)

Link: https://lore.kernel.org/r/20190813204513.4790-2-skunberg.kelsey@gmail.comSigned-off-by: Kelsey Skunberg <skunberg.kelsey@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Donald Dutile <ddutile@redhat.com>

The comment describing pci_p2pdma_distance_many() still referred to
the devices being behind the same root port. This no longer applies
so reword the documentation.

Link: https://lore.kernel.org/r/20190730163545.4915-15-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-15-logang@deltatee.comSigned-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

Now that we map the requests correctly we can remove the iommu_present()
restriction.

Link: https://lore.kernel.org/r/20190730163545.4915-14-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-14-logang@deltatee.comSigned-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

Any requests that traverse the host bridge will need to be mapped into the
IOMMU, so call dma_map_sg() inside pci_p2pdma_map_sg() when appropriate.

Similarly, call dma_unmap_sg() inside pci_p2pdma_unmap_sg().

Link: https://lore.kernel.org/r/20190730163545.4915-13-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-13-logang@deltatee.comSigned-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

When upstream_bridge_distance() is called, store the method required to map
the DMA transfers in an xarray so it can be looked up efficiently on the
hot path in pci_p2pdma_map_sg().

Link: https://lore.kernel.org/r/20190730163545.4915-12-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-12-logang@deltatee.comSigned-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

Factor out the bus-only mapping into its own static function.  No
functional changes. The original pci_p2pdma_map_sg_attrs() will be used to
decide whether this is an appropriate way to map.

Link: https://lore.kernel.org/r/20190730163545.4915-11-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-11-logang@deltatee.comSigned-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

Add pci_p2pdma_unmap_sg() to the two places that call pci_p2pdma_map_sg().

This is a prep patch to introduce correct mappings for p2pdma transactions
that go through the root complex.

Link: https://lore.kernel.org/r/20190730163545.4915-10-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-10-logang@deltatee.comSigned-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

This is to match the dma_map_sg() API which this function will have to call
in an future patch.

Add a pci_p2pdma_map_sg_attrs() function and helper to call it with no
attributes just like the dma_map_sg() function.

Link: https://lore.kernel.org/r/20190730163545.4915-9-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-9-logang@deltatee.comSigned-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

Intel devices do not have good support for P2P requests that span different
host bridges as the transactions will cross the QPI/UPI bus and this does
not perform well.

Therefore, enable support for these devices only if the host bridges match.

Add Intel devices that have been tested and are known to work. There are
likely many others out there that will need to be tested and added.

Link: https://lore.kernel.org/r/20190730163545.4915-8-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-8-logang@deltatee.comSigned-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

Push both PCI devices into the whitelist checking function seeing some
hardware will require us ensuring they are on the same host bridge.

At the same time we rename root_complex_whitelist() to
host_bridge_whitelist() to match the terminology used in the code.

Link: https://lore.kernel.org/r/20190730163545.4915-7-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-7-logang@deltatee.comSigned-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>

When a P2PDMA transfer is rejected due to ACS being set, we can also check
the whitelist and allow the transactions.

Do this by pushing the whitelist check into the upstream_bridge_distance()
function.

Link: https://lore.kernel.org/r/20190730163545.4915-6-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-6-logang@deltatee.comSigned-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>