PCIe

PCIe uses a bidirectional connection to send and receive information at the same time, as shown in the following figure.

Figure: PCIe device connection link

The path between the devices is called a Link. It is made up of one or more transmit and receive pairs. One pair of the Link is called a Lane. The PCIe device connection in Qualcomm Linux devices supports 16 lanes. The number of lanes or the Link width is x16. The following table lists the types of PCIe connections for devices.

Table : PCIe connections

PCIe type	Description
Root complex (RC)	Connects the CPU to the PCIe topology
Switch	Connects more than 2 ports and acts as a packet router
Bridge	Connects different buses: for example, PCIe to PCIe, or PCIe to peripheral component interconnect (PCI)
Endpoint (EP)	Resides at the bottom of the PCIe topology tree structure and has only an upstream port
Legacy endpoint	Uses older PCI bus operations to support backward compatibility

PCIe host mode enumeration feature

When a system first powers up, the configuration software running on the system host processor is aware of the existence of only Bus 0 (if PCIe is supported). The software is not aware of the bus topology or any device connected to the bus. The enumeration process discovers the various buses, devices, and functions present in the system. When enumeration is complete, each bus in the system is numbered as follows:

The primary bus number indicates the bus that directly connects to the primary interface of the bridge (towards the root complex).
The secondary bus number indicates the bus that directly connects to the secondary interface of the bridge (away from the root complex).
The subordinate bus number indicates the highest numbered bus that exists on the downstream side of the bridge.

The BDF number uniquely identifies each device. The ID-based routing method of the transaction layer packet (TLP) uses this number. The PCIe host mode enumeration process involves the following:

Link training
Scanning for devices on the bus
Registration

For more information on the PCIe device initialization, enumeration process, see https://www.kernel.org/doc/html/latest/PCI/index.html.

PCIe layered architecture

The following figure shows the PCIe software architecture.

Figure : PCIe software architecture

The following figure shows the layered architecture model of PCIe.

Figure : PCIe layered architecture

The transmission units exchanged are as follows.

Ordered set between the physical layer entities.
Data link layer packet (DLLP) between data link layer entities.
Transaction layer packet (TLP) between transaction layer entities.

The following table lists the three layers, with the respective functions, in the PCIe architecture.

Table : Layers in PCIe architecture

Layer	Features
Physical layer	Logical sub-block: Link training, initialization, and maintenance.
Physical layer	Physical sub-block: 8b/10b encoding and decoding, and parallel-to-serial and serial-to-parallel conversion.
Data link layer	Assembly and disassembly of the DLLP packet.
	Generation and validation of the link layer CRC (LCRC).
	Acknowledgment and no acknowledgment protocol (replay of TLPs in error).
Transaction layer	Assembly and disassembly of the TLP packet.
	Generation and validation of end-to-end CRC (ECRC).
	Flow control receives entity advertises for the available receive buffer size information using DLLPs.
	Quality of service (QoS): traffic class (TC) to virtual channel (VC) mapping.
	Transaction ordering: implements the transaction ordering rule within a VC.

PCI defines a dedicated block of configuration address space for each function as shown in the following figure. The software determines the presence of a function, configures it, and checks and controls its status.

Figure : PCIe configuration address space

PCIe software driver configuration

The PCIe controller driver initializes the PCIe resources and performs link training. After successful training, the controller driver calls the PCIe framework for link enumeration, such as endpoint discovery, identifying the client driver, and probing those drivers. For more information about the PCIe framework and client driver PCIe registrations, see https://www.kernel.org/doc/html/latest/PCI/index.html.

Link training

Link training comprises the following operations:

The PCIe driver pcie-qcom.c file at https://github.com/torvalds/linux/blob/master/drivers/pci/controller/dwc/pcie-qcom.c obtains the required resources such as regulators, clocks, from the device tree.
The PCIe driver calls Synopsys DesignWare® Core host driver pcie-designware-host.c file at https://github.com/torvalds/linux/blob/master/drivers/pci/controller/dwc/pcie-designware-host.c to initialize the root complex.
The Synopsys DesignWare Core driver performs all necessary initializations.
The Synopsys DesignWare Core driver calls a function pointer to perform host initialization.
The Qualcomm PCIe driver performs PHY power-on, enables all regulators, clocks.
The Synopsys DesignWare Core driver starts the link training by calling the function pointer to start the link.

Hardware initialization

The driver initializes and configures the PCIe hardware block and performs link training. The initialization occurs after the platform _probe() driver function is called.

static int qcom_pcie_probe(struct platform_device *pdev)
{

        // get PCIe resources

        ret = dw_pcie_host_init(pp);  // call DWC framework to host intialisation like MSI, MSIx and controller init 
        if (ret) {
                dev_err(dev, "cannot initialize host\n");
                goto err_phy_exit;
        }

};

static const struct of_device_id qcom_pcie_match[] = {
        { .compatible = "qcom,pcie-sc7280", .data = &cfg_1_9_0 },
        {}

};

static struct platform_driver qcom_pcie_driver = {
        .probe = qcom_pcie_probe,
        .driver = {
                .name = "qcom-pcie",
                .suppress_bind_attrs = true,
                .of_match_table = qcom_pcie_match,
        },
};

When the Synopsys DesignWare Core driver is initialized, it also initializes MSI, MSIx, and controllers. The driver calls a Qualcomm function pointer to start the link training.

static int qcom_pcie_start_link(struct dw_pcie *pci)
{
        struct qcom_pcie *pcie = to_qcom_pcie(pci);

        /* Enable Link Training state machine */
        if (pcie->cfg->ops->ltssm_enable)
                pcie->cfg->ops->ltssm_enable(pcie);

        return 0;
}

static int qcom_pcie_link_up(struct dw_pcie *pci)
{
        u16 offset = dw_pcie_find_capability(pci, PCI_CAP_ID_EXP);
        u16 val = readw(pci->dbi_base + offset + PCI_EXP_LNKSTA);

        return !!(val & PCI_EXP_LNKSTA_DLLLA); // checks the link is up or not
}
static const struct dw_pcie_ops dw_pcie_ops = {
        .link_up = qcom_pcie_link_up,
        .start_link = qcom_pcie_start_link,
};

The Synopsys DesignWare Core driver waits for the link to be active to enumerate the PCI framework. The Qualcomm PCIe driver enables only the link training.

Note: The PCIE_0 root complex instance is enabled by default for the WLAN EP connection.

Enable QPS615 PCIe switch

This section describes how to enable a QPS615 PCIe switch in the Qualcomm Linux hardware SoCs. The QPS615 switch endpoint is supported on the PCIe1 instance. The following figure shows the QPS615 endpoint and connections.

Figure : QPS615 PCIe switch connection diagram

The Qualcomm PCIe driver documentation can be accessed at the following locations:

The following configurations are enabled by default to support the QPS615 switch. Disable the QPS615 switch default support, by reverting the code changes, to use it for a different PCIe endpoint. To enable PCIe-related configs, apply the following patch to the /arch/arm64/configs/qcom_addons.config file.

QCLINUX: arm64: defconfig: qcom:
diff --git a/arch/arm64/configs/qcom_defconfig b/arch/arm64/configs/qcom_defconfig
index 7293680..730122c 100644
--- a/arch/arm64/configs/qcom_defconfig
+++ b/arch/arm64/configs/qcom_defconfig
@@ -173,6 +173,7 @@
 CONFIG_HOTPLUG_PCI=y
 CONFIG_HOTPLUG_PCI_ACPI=y
 CONFIG_PCI_HOST_GENERIC=y
+CONFIG_PCIE_DW_PLAT_HOST=y
 CONFIG_PCIE_QCOM=y
 CONFIG_PCI_ENDPOINT=y
 CONFIG_PCI_ENDPOINT_CONFIGFS=y
@@ -793,3 +794,5 @@
 CONFIG_CORESIGHT_TPDM=m
 CONFIG_CORESIGHT_DUMMY=m
 CONFIG_MEMTEST=y
+CONFIG_PCIEASPM=y
+CONFIG_PCIEASPM_POWER_SUPERSAVE=y

Message signaled interrupt (MSI)

The current MSI mapping doesn’t have all the vectors. The Qualcomm Linux hardware SoCs support eight vectors. Each vector in turn supports 32 MSIs. Therefore, the total MSIs supported are 256. For information about adding all the MSI groups supported for this PCIe instance, see https://lore.kernel.org/linux-arm-msm/f1168212-bc6e-4570-869c-2870d6f248ad@linaro.org/T/.

Sample PCIe kernel driver log

The following is a sample PCIe kernel driver log from the QPS615 device enumeration.

Ethernet interfaces supported through QPS615 PCIe switch

The QPS615 PCIe switch enables Ethernet connectivity for the device. When the device loads the QPS615 driver and establishes the PCIe link, it automatically activates the supported Ethernet interfaces, by default, during device startup. To customize the default configuration or enable extra MAC/PHY components beyond Qualcomm’s hardware setup, see the Bring up Ethernet section in the Qualcomm Linux Ethernet guide.

Table : Supported Ethernet interfaces

Interface type	Speed	Connector type	Description
QEP PHY (SGMII)	2.5 GbE	IX/RJ45 connector (QEP8121)	Enabled by default.
AQR PHY (USXGMII)	10 GbE	IX/RJ45 connector (AQR113C)	The 10 Gigabit Ethernet AQR PHY is an optional component in the RB3 Gen 2 Development Kit, and the Evaluation Kits (EVK) for Dragonwing IQ-8275 and Dragonwing IQ-9075. It’s validated on AQR113C.

Bring up alternate hardware components

You can attach MAC/PHY components other than the hardware configuration provided by Qualcomm and bring them up. To replace QPS615 with other PCIe based MAC/PHY, see Enable QPS615 PCIe switch.

Note: You must obtain the MAC/PHY driver and firmware from the respective vendor. Qualcomm isn’t responsible for these configuration changes.

Enable USB interface through PCIe switch

This section provides instructions on how to activate a USB interface through a PCIe switch in the Qualcomm Linux hardware SoCs. The PCIE1 instance is connected to the endpoint of the QPS615 switch, and the downstream port of the QPS615 is connected to the PCIe to USB endpoint. For the PCIe to USB endpoint connections using the QPS615, see the mainboard and interposer block diagram at https://docs.qualcomm.com/bundle/publicresource/topics/80-80021-251/rb3_hardware_overview.html.

Note: Dragonwing IQ-9075 PCIe software doesn’t support USB.

Power to usb hub is enabled through pwrctrl driver, Please refer below upstream links for these details. https://elixir.bootlin.com/linux/v7.1/source/arch/arm64/boot/dts/qcom/qcs6490-rb3gen2.dts#L896 https://elixir.bootlin.com/linux/v7.1/source/Documentation/devicetree/bindings/usb/renesas,upd720201-pci.yaml https://elixir.bootlin.com/linux/v7.1/source/drivers/pci/pwrctrl/generic.c

### **Avoid early handoff in PCIe to USB controller**

Ensure that `usb_early-handoff` is skipped for the PCIe to USB controller since the firmware isn't loaded. This precaution is necessary to prevent any adverse impact on the bootup duration.

```text
diff --git a/drivers/usb/host/pci-quirks.c b/drivers/usb/host/pci-quirks.c
index 2665832..c3d488f 100644
--- a/drivers/usb/host/pci-quirks.c
+++ b/drivers/usb/host/pci-quirks.c
@@ -1260,6 +1260,11 @@
 			return;
 	}
 
+	/* Skip handoff for Renesas PCI USB controller on QCOM SOC */
+	if ((pdev->vendor == PCI_VENDOR_ID_RENESAS) &&
+		(pcie_find_root_port(pdev)->vendor == PCI_VENDOR_ID_QCOM))
+		return;
+
 	if (pdev->class != PCI_CLASS_SERIAL_USB_UHCI &&
 			pdev->class != PCI_CLASS_SERIAL_USB_OHCI &&
 			pdev->class != PCI_CLASS_SERIAL_USB_EHCI &&

Download PCIe to USB controller firmware

To download the firmware from https://www.renesas.com/us/en/products/interface/usb-switches-hubs/upd720201-usb-30-host-controller#design_development, register and log in to https://www.renesas.com/. Rename the downloaded firmware file to renesas_usb_fw.mem.

Note: To prevent command failures, update the software as described in the Set up the device section before updating the Renesas firmware.

Option 1 (Ubuntu): Push the firmware files using SCP or similar tools. For example,

scp renesas_usb_fw.mem root@10.92.175.138:/lib/firmware

Option 2 (Yocto):

Connect device to Host PC via USB cable for adb.

Push firmware to device.

mount -o remount,rw /usr
adb push renesas_usb_fw.mem /lib/firmware

To activate the firmware do either of the following options.
- Option A: Reboot target for USB type A ports.
- Option B: Manually bind the Renesas xHCI driver.
  echo "0001:04:00.0" > /sys/bus/pci/drivers/xhci-pci-renesas/bind
Verify firmware enumeration.
dmesg

PCIe kernel driver logs for PCIe to USB device enumeration reference

You can run the following commands to view the device information:

To display device information in USB, run the following command.

lsusb

The following message is displayed.

Bus 002 Device 003: ID 0b95:1790 ASIX Electronics Corp. AX88179 Gigabit Ethernet
Bus 002 Device 002: ID 05e3:0625 Genesys Logic, Inc. USB3.2 Hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 05e3:0610 Genesys Logic, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

To display device information in PCIe, run the following command.

lspci

The following message is displayed.

00:00.0 PCI bridge: Qualcomm Device 010b
01:00.0 PCI bridge: Toshiba Corporation Device 0623
02:01.0 PCI bridge: Toshiba Corporation Device 0623
02:02.0 PCI bridge: Toshiba Corporation Device 0623
02:03.0 PCI bridge: Toshiba Corporation Device 0623
04:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller (rev 03)
05:00.0 Ethernet controller: Toshiba Corporation Device 0220
05:00.1 Ethernet controller: Toshiba Corporation Device 0220

Connect QPS615 switches in cascade

Connect the QPS615 switches in cascade to enable additional Ethernet, PCIe, and USB ports.

Note: This feature is supported only in QCS6490.

The following image shows the two QPS615 switches connected in cascade to PCIE1 and another QPS615 switch connected to PCIE2.

QPS615 cascade connection — Figure : QPS615 switches in cascade connection

To initiate the link training and enumeration for all endpoints, do the following:

To reset the QPS615 switch, toggle the RESX GPIOs for both QPS615 #1 and QPS615 #2.
To control the endpoint reset, trigger PERST. Both the switches share the PERST.

The devices can be attached directly to the QPS615 switch. The PCIe RC0 is attached to the WLAN endpoint. The following differences are due to the PCIe node hierarchy.

Switch-attached devices:
- The QPS615 PCIe tree node hierarchy is statically fixed.
- All nodes for switching USP and DSP ports are created during PCIe initialization.
- One of the switch DSP ports represents WLAN.
- If you disable the WLAN node, it disables the WLAN device, but the PCIe downstream port remains enabled and returns a default maximum link width.
Directly-attached WLAN devices:
- Disables only a single node.
- Returns the Invalid argument, when WLAN is disabled.

Enable NVMe through PCIe interface

This section describes how to enable NVMe using PCIe for storage expansion. To verify if NVMe is connected over a PCIe interface, do the following:

To display PCIe device information, run the following command.

lspci

Output:

00:00.0 PCI bridge: Qualcomm Device 0115
01:00.0 Network controller: Qualcomm QCNFA765 Wireless Network Adapter (rev 01)
00:00.0 PCI bridge: Qualcomm Device 0115
01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO

Locate the PCIe logs.

dmesg | grep pcie

Output:

[    0.000000] Kernel command line:  root=/dev/disk/by-partlabel/system rw rootwait console=ttyMSM0,115200n8 
pcie_pme=nomsi net.ifnames=0 pci=noaer kpti=off kasan=off kasan.stacktrace=off swiotlb=128 earlycon reboot=panic_warm page_owner=on qcom_scm.download_mode=1 slub_debug=FZP,
zs_handle,zspage;FZPU mitigations=auto kernel.sched_pelt_multiplier=4 rcupdate.rcu_expedited=1 rcu_nocbs=0-7 no-steal-acc vfio_iommu_type1.allow_unsafe_interrupts=1 fw_devlink.strict=1
	[    3.842071] qcom-pcie 1c00000.pci: supply vdda not found, using dummy regulator
	[    3.845339] qcom-pcie 1c10000.pci: supply vdda not found, using dummy regulator
	[    3.845428] qcom-pcie 1c10000.pci: supply vddpe-3v3 not found, using dummy regulator
	[    3.845508] qcom-pcie 1c10000.pci: host bridge /pci@1c10000 ranges:
	[    3.845519] qcom-pcie 1c10000.pci:       IO 0x0060200000..0x00602fffff -> 0x0000000000
	[    3.845526] qcom-pcie 1c10000.pci:      MEM 0x0060300000..0x007fffffff -> 0x0060300000
	[    3.848154] qcom-pcie 1c00000.pci: supply vddpe-3v3 not found, using dummy regulator
	[    3.854556] qcom-pcie 1c00000.pci: host bridge /pci@1c00000 ranges:
	[    3.860574] qcom-pcie 1c00000.pci:       IO 0x0040200000..0x00402fffff -> 0x0000000000
	[    3.867362] qcom-pcie 1c00000.pci:      MEM 0x0040300000..0x005fffffff -> 0x0040300000
	[    3.963275] qcom-pcie 1c10000.pci: iATU: unroll T, 32 ob, 8 ib, align 4K, limit 1024G
	[    3.990385] qcom-pcie 1c00000.pci: iATU: unroll T, 32 ob, 8 ib, align 4K, limit 1024G
	[    4.072091] qcom-pcie 1c10000.pci: PCIe Gen.4 x2 link up
	[    4.080241] qcom-pcie 1c10000.pci: PCI host bridge to bus 0001:00
	[    4.291824] pcieport 0001:00:00.0: Adding to iommu group 9
	[    4.551677] pcieport 0001:00:00.0: PME: Signaling with IRQ 256
	[    4.572484] WARNING: CPU: 3 PID: 89 at drivers/pci/controller/dwc/pcie-qcom.c:1430 qcom_pcie_icc_update+0xf8/0x144
	[    4.595291] pc : qcom_pcie_icc_update+0xf8/0x144
	[    4.595293] lr : qcom_pcie_icc_update+0x9c/0x144
	[    4.640742]  qcom_pcie_icc_update+0xf8/0x144
	[    4.640745]  qcom_pcie_probe+0x234/0x30c
	[    4.988519] qcom-pcie 1c00000.pci: Phy link never came up
	[    5.302179] qcom-pcie 1c00000.pci: PCI host bridge to bus 0000:00
	[    5.372008] pcieport 0000:00:00.0: Adding to iommu group 10
	[    5.378905] pcieport 0000:00:00.0: PME: Signaling with IRQ 257
	[   11.564590] pcieport 0000:00:00.0: BAR 14: assigned [mem 0x40400000-0x405fffff]

To locate the NVMe directories, run the following command.

ls -lh /dev/nvme*

Output:

crw-------. 1 root root 508,  0 Jan  1 00:00 /dev/nvme0
	brw-rw----. 1 root disk 259, 30 Jan  1 00:00 /dev/nvme0n1
	

PCIe client driver sample

The client driver defines the device-id table and pci_driver structures, and registers with the PCIe framework. The following are a few PCIe client driver samples for reference.

Sample data structure to hold client-specific private data.

struct sample_driver_data {
    int driver_data;
};

Sample driver: You can provide data according to your driver-specific data structure.
struct sample_driver_data info = {};

Sample device ID table with the driver-specific data. The client driver registers with the 0x306 device ID.

static const struct pci_device_id sample_pci_id_table[] = {
    { PCI_DEVICE_SUB(PCI_VENDOR_ID_QCOM, 0x0306, PCI_VENDOR_ID_QCOM, 0x010c),
        .driver_data = (kernel_ulong_t) &info },
    {  } 
};
MODULE_DEVICE_TABLE(pci, sample_pci_id_table);

Note: MODULE_DEVICE_TABLE(pci, sample_pci_id_table); is mandatory.

Sample pci_driver data structure with client driver name, pci-id table, and callbacks. The pointer to this structure is passed while registering with the PCI frame work.

static struct pci_driver sample_pci_driver = {
      .name           = "Sample-PCI-Client-driver",        /* Give the name that suites your driver description */
      .id_table       = sample_pci_id_table,               /* pci core driver will compare from ids's supplied in this table with the enumerated endpoints */
      .probe          = sample_pci_probe,                  /* probe function is invoked by the PCI f/w when id matches with the endpoint id's */
      .remove         = sample_pci_remove,                 /* Remove function is invoked by PCI f/w when senses the attached endpoint/function is detached or down */
      .driver.pm      = &sample_pci_pm_ops                 /* pm ops that will be called from the pci core driver*/
};

To register with PCI firmware, call pci_register_driver(&sample_pci_driver) from module_init().
module_pci_driver(sample_pci_driver);

PCIe bringup

For information about PCIe bringup, see PCIe-related configurations and QPS615 switch support.

PCIe power optimization

PCIe defines two types of power management methods.

Power management software that determines the power management capability of each device and manages each device individually
System that doesn’t require software intervention such as active state power management (ASPM)

During a time period when no packet is transmitted through the link, a device places the link into a power-saving state.

PCIe L0 link states

PCIe power management defines the following L0 link states:

L0: active state where all PCIe transactions and other operations are enabled
L0s: ASPM state with low-resume latency (energy saving standby state)

PCIe device states

PCIe power management defines the following device states:

D0 (mandatory): The device is in full ON state, where there are two substates
- D0_{uninitialized}: The function is present in the D0_{uninitialized} state after the device comes out of reset, waiting to be enumerated and configured.
- D0_active
  - The function is present in the D0_active state following the completion of the enumeration and configuration process.
  - The function enters the D0_active state when the system software enables one or more (in any combination) function parameters, such as memory space enable, I/O space enable, or BME bits.
D1 (optional): light-sleep state
- The function can’t initiate a TLP except for the PME message
- The function can’t act as the target of transactions other than for configuration transactions.
- The function issues a software command to enter the D1 state by programming the PM control and status register.
D2 (optional): deep-sleep state
- The function can’t initiate a TLP except for the PME message
- The function can’t act as the target of transactions other than configuration transactions.
- The function issues a software command to enter the D2 state by programming the PM control and status register.
D3 (mandatory): device is the lowest power state, where the function must support both the D3 states
D3_hot
- The function can’t initiate a TLP except for the PME message.
- The function can’t act as the target of transactions other than configuration transactions.
- The function issues a software command to enter the D3_hot state by programming the power state field.
D3_cold: device enters the D3_cold state and power is removed; when power is restored, the device enters the D0_{uninitialized} state.

PCIe verification

For information about PCIe verification, see PCIe-related configurations and QPS615 switch support.

Debug PCIe issues

The lspci and setpci commands are native to Linux distributions. These commands have various levels of output. These commands also provide a useful point-in-time look at the capabilities and status of the different components trained on the PCI bus. Most of these capabilities are reflections of the configuration space registers required by the PCIe base specification. For more details, see https://pcisig.com/specifications. To view the usage instructions, run the following command.

lspci --help

The following features are useful in troubleshooting PCIe issues.

Display device information

lspci

The following message is displayed.

00:00.0 PCI bridge: Qualcomm Device 010b
01:00.0 PCI bridge: Toshiba Corporation Device 0623
02:01.0 PCI bridge: Toshiba Corporation Device 0623
02:02.0 PCI bridge: Toshiba Corporation Device 0623
02:03.0 PCI bridge: Toshiba Corporation Device 0623
04:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller (rev 03)
05:00.0 Ethernet controller: Toshiba Corporation Device 0220
05:00.1 Ethernet controller: Toshiba Corporation Device 0220

Display PCIe device and vendor IDs in the device control register.

lspci -nvmm

The following message is displayed.

Slot:   0001:00:00.0
Class:  0604
Vendor: 17cb
Device: 010b
DTNode: /sys/firmware/devicetree/base/soc@0/pci@1c08000/pcie@0
IOMMUGroup:     33
 
Slot:   0001:01:00.0
Class:  0604
Vendor: 1179
Device: 0623
DTNode: /sys/firmware/devicetree/base/soc@0/pci@1c08000/pcie@0/pcie1_bus1_dev0_fn0
IOMMUGroup:     33
 
Slot:   0001:02:01.0
Class:  0604
Vendor: 1179
Device: 0623
DTNode: /sys/firmware/devicetree/base/soc@0/pci@1c08000/pcie@0/pcie1_bus1_dev0_fn0/pcie1_bus2_dev1_fn0
IOMMUGroup:     33
 
Slot:   0001:02:02.0
Class:  0604
Vendor: 1179
Device: 0623
DTNode: /sys/firmware/devicetree/base/soc@0/pci@1c08000/pcie@0/pcie1_bus1_dev0_fn0/pcie1_bus2_dev2_fn0
IOMMUGroup:     33
 
Slot:   0001:02:03.0
Class:  0604
Vendor: 1179
Device: 0623
DTNode: /sys/firmware/devicetree/base/soc@0/pci@1c08000/pcie@0/pcie1_bus1_dev0_fn0/pcie1_bus2_dev3_fn0
IOMMUGroup:     33
 
Slot:   0001:04:00.0
Class:  0c03
Vendor: 1912
Device: 0014
Rev:    03
ProgIf: 30
DTNode: /sys/firmware/devicetree/base/soc@0/pci@1c08000/pcie@0/pcie1_bus1_dev0_fn0/pcie1_bus2_dev2_fn0/pcie1_bus4_dev0_fn0
IOMMUGroup:     33
 
Slot:   0001:05:00.0
Class:  0200
Vendor: 1179
Device: 0220
SVendor:        1179
SDevice:        0001
DTNode: /sys/firmware/devicetree/base/soc@0/pci@1c08000/pcie@0/pcie1_bus1_dev0_fn0/pcie1_bus2_dev3_fn0/qps615_eth0,qps615_eth0@pcie1_rp
IOMMUGroup:     33
 
Slot:   0001:05:00.1
Class:  0200
Vendor: 1179
Device: 0220
SVendor:        1179
SDevice:        0001
DTNode: /sys/firmware/devicetree/base/soc@0/pci@1c08000/pcie@0/pcie1_bus1_dev0_fn0/pcie1_bus2_dev3_fn0/qps615_eth1,qps615_eth1@pcie1_rp
IOMMUGroup:     33

For more info on PCIe debugging, see https://www.kernel.org/doc/html/v4.17/driver-api/pci.html.

PCIe examples

For information about the upstream device tree reference, see the following files.

For information about device-tree node for the Qualcomm Linux hardware SoCs, see the following DTSI files.

QCS6490 and QCS5430: https://elixir.bootlin.com/linux/v7.0-rc7/source/arch/arm64/boot/dts/qcom/kodiak.dtsi
Dragonwing IQ-9075: https://elixir.bootlin.com/linux/v7.0-rc7/source/arch/arm64/boot/dts/qcom/lemans.dtsi
Dragonwing IQ-615: https://git.kernel.org/pub/scm/linux/kernel/git/qcom/linux.git/tree/arch/arm64/boot/dts/qcom/qcs615.dtsi?h=arm64-for-6.16

Client and PCI driver operation flow example

The following figure shows the sequence that the PCIe client driver follows to configure the PCIe driver for a client.

Client and PCI driver high-level call flow example

The following figure shows the high-level call flow and call details between the PCIe client driver and PCIe driver.

Introduction

Architecture

Key Documents

Technologies

System

Release Notes

PCIe host mode enumeration feature

PCIe layered architecture

PCIe software driver configuration

Link training

Hardware initialization

Enable QPS615 PCIe switch

Message signaled interrupt (MSI)

Sample PCIe kernel driver log

Ethernet interfaces supported through QPS615 PCIe switch

Bring up alternate hardware components

Enable USB interface through PCIe switch

Download PCIe to USB controller firmware

PCIe kernel driver logs for PCIe to USB device enumeration reference

Connect QPS615 switches in cascade

Enable NVMe through PCIe interface

PCIe client driver sample

PCIe bringup

PCIe power optimization

PCIe L0 link states

PCIe device states

PCIe verification

Debug PCIe issues

PCIe examples

Client and PCI driver operation flow example

Client and PCI driver high-level call flow example

​PCIe host mode enumeration feature

​PCIe layered architecture

​PCIe software driver configuration

​Link training

​Hardware initialization

​Enable QPS615 PCIe switch

​PCIe-related configurations

​Message signaled interrupt (MSI)

​Sample PCIe kernel driver log

​Ethernet interfaces supported through QPS615 PCIe switch

​Bring up alternate hardware components

​Enable USB interface through PCIe switch

​Download PCIe to USB controller firmware

​PCIe kernel driver logs for PCIe to USB device enumeration reference

​Connect QPS615 switches in cascade

​Enable NVMe through PCIe interface

​PCIe client driver sample

​PCIe bringup

​PCIe power optimization

​PCIe L0 link states

​PCIe device states

​PCIe verification

​Debug PCIe issues

​PCIe examples

​Client and PCI driver operation flow example

​Client and PCI driver high-level call flow example

PCIe host mode enumeration feature

PCIe layered architecture

PCIe software driver configuration

Link training

Hardware initialization

Enable QPS615 PCIe switch

PCIe-related configurations

Message signaled interrupt (MSI)

Sample PCIe kernel driver log

Ethernet interfaces supported through QPS615 PCIe switch

Bring up alternate hardware components

Enable USB interface through PCIe switch

Download PCIe to USB controller firmware

PCIe kernel driver logs for PCIe to USB device enumeration reference

Connect QPS615 switches in cascade

Enable NVMe through PCIe interface

PCIe client driver sample

PCIe bringup

PCIe power optimization

PCIe L0 link states

PCIe device states

PCIe verification

Debug PCIe issues

PCIe examples

Client and PCI driver operation flow example

Client and PCI driver high-level call flow example