PCI/DPC: Await readiness of secondary bus after reset

commit 53b54ad074de1896f8b021615f65b27f557ce874 upstream.

pci_bridge_wait_for_secondary_bus() is called after a Secondary Bus
Reset, but not after a DPC-induced Hot Reset.

As a result, the delays prescribed by PCIe r6.0 sec 6.6.1 are not
observed and devices on the secondary bus may be accessed before
they're ready.

One affected device is Intel's Ponte Vecchio HPC GPU.  It comprises a
PCIe switch whose upstream port is not immediately ready after reset.
Because its config space is restored too early, it remains in
D0uninitialized, its subordinate devices remain inaccessible and DPC
recovery fails with messages such as:

  i915 0000:8c:00.0: can't change power state from D3cold to D0 (config space inaccessible)
  intel_vsec 0000:8e:00.1: can't change power state from D3cold to D0 (config space inaccessible)
  pcieport 0000:89:02.0: AER: device recovery failed

Fix it.

Link: https://lore.kernel.org/r/9f5ff00e1593d8d9a4b452398b98aa14d23fca11.1673769517.git.lukas@wunner.de
Tested-by: Ravi Kishore Koppuravuri <ravi.kishore.koppuravuri@intel.com>
Signed-off-by: Lukas Wunner <lukas@wunner.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Mika Westerberg <mika.westerberg@linux.intel.com>
Cc: stable@vger.kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index f1a3f165f88af7c33cc8d6110eee093716a3b9e2..d37013d007b6eff877def47210edf8fc0401bdc7 100644 (file)
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -157,9 +157,6 @@ static int __init pcie_port_pm_setup(char *str)
 }
 __setup("pcie_port_pm=", pcie_port_pm_setup);
 
-/* Time to wait after a reset for device to become responsive */
-#define PCIE_RESET_READY_POLL_MS 60000
-
 /**
  * pci_bus_max_busnr - returns maximum PCI bus number of given bus' children
  * @bus: pointer to PCI bus structure to search

diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 77dd7bbe861d662d8c2ba89f44d4fea0a42c84e7..72436000ff2525cb11c00e4e74e1f7ed5705d92a 100644 (file)
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -53,6 +53,12 @@ int pci_bus_error_reset(struct pci_dev *dev);
  * Reset (PCIe r6.0 sec 5.8).
  */
 #define PCI_RESET_WAIT         1000    /* msec */
+/*
+ * Devices may extend the 1 sec period through Request Retry Status completions
+ * (PCIe r6.0 sec 2.3.1).  The spec does not provide an upper limit, but 60 sec
+ * ought to be enough for any device to become responsive.
+ */
+#define PCIE_RESET_READY_POLL_MS 60000 /* msec */
 
 /**
  * struct pci_platform_pm_ops - Firmware PM callbacks

diff --git a/drivers/pci/pcie/dpc.c b/drivers/pci/pcie/dpc.c
index c556e7beafe38d8e26c59906df74f371a13cf6a9..f21d64ae4ffccf05abbdb17e1ccdd61f14b6375c 100644 (file)
--- a/drivers/pci/pcie/dpc.c
+++ b/drivers/pci/pcie/dpc.c
@@ -170,8 +170,8 @@ pci_ers_result_t dpc_reset_link(struct pci_dev *pdev)
        pci_write_config_word(pdev, cap + PCI_EXP_DPC_STATUS,
                              PCI_EXP_DPC_STATUS_TRIGGER);
 
-       if (!pcie_wait_for_link(pdev, true)) {
-               pci_info(pdev, "Data Link Layer Link Active not set in 1000 msec\n");
+       if (pci_bridge_wait_for_secondary_bus(pdev, "DPC",
+                                             PCIE_RESET_READY_POLL_MS)) {
                clear_bit(PCI_DPC_RECOVERED, &pdev->priv_flags);
                ret = PCI_ERS_RESULT_DISCONNECT;
        } else {