xio3130-downstream: missing x-speed/x-width properties forces PHB topology for VFIO GPU+NIC passthrough, eliminating PIX path required for AI/ML collective performance

Environment

• QEMU: 10.1.0 (qemu-kvm-10.1.0-17.el9_8.5, el9)
• Host: baremetal x86_64, VFIO passthrough into KubeVirt VMs
• Guest OS: RHCOS 9.8 / Linux 5.14.0-687 (OpenShift 4.22)
• Devices: 8× NVIDIA H100 SXM5 (PF passthrough) + 8× Mellanox CX7 NDR 400G (PF passthrough) per VM

Problem

xio3130-downstream does not expose x-speed or x-width device properties. This makes it impossible to simultaneously achieve:

1. PIX topology (GPU and NIC behind the same emulated switch — required for optimal GPU→NIC DMA in AI/ML workloads)
2. Correct PCIe link speed advertisement (Gen4/Gen5)

Attempting -global xio3130-downstream.x-speed=32 causes a fatal launch error: Property 'xio3130-downstream.x-speed' not found

pcie-root-port exposes these properties via the PCIESlot class; xio3130-downstream does not inherit them.

Root Cause

pcie_cap_v1_fill() in hw/pci/pcie.c (lines 97–106) hardcodes LNKCAP/LNKSTA to 2.5GT/s x1:

pci_set_long(exp_cap + PCI_EXP_LNKCAP,
             QEMU_PCI_EXP_LNKCAP_MLW(QEMU_PCI_EXP_LNK_X1) |
             QEMU_PCI_EXP_LNKCAP_MLS(QEMU_PCI_EXP_LNK_2_5GT));

pci_set_word(exp_cap + PCI_EXP_LNKSTA,
             QEMU_PCI_EXP_LNKSTA_NLW(QEMU_PCI_EXP_LNK_X1) |
             QEMU_PCI_EXP_LNKSTA_CLS(QEMU_PCI_EXP_LNK_2_5GT));

hw/pci-bridge/xio3130_upstream.c:81 calls pcie_cap_init() with no speed override.
The x-speed/x-width properties exist on PCIESlot (pcie-root-port) but xio3130 does not inherit this class.

Result inside guest after injecting xio3130 switch:
$ lspci -vv -s 82:04.0   # xio3130-downstream with NIC PF
LnkCap: Speed 2.5GT/s, Width x1
LnkSta: Speed 2.5GT/s (ok), Width x1 (ok)   ← hardcoded, cannot be overridden

$ lspci -vv -s 87:00.0   # NIC PF itself
LnkSta: Speed 32GT/s (ok), Width x16 (ok)   ← real hardware speed

Impact on AI/ML Workloads

NCCL (NVIDIA Collective Communications Library) reads guest PCIe topology to determine:
1. GPU↔NIC path type: PIX (one switch hop) vs PHB (host bridge) — PIX is required for optimal GPU→NIC DMA
2. maxBw per channel — derived from PCIe link parameters along the path

The lack of x-speed/x-width on xio3130-downstream forces an impossible choice:

┌──────────────────────┬───────────┬────────────┬────────────┬──────────────────────┐
│    QEMU topology     │ Path type │   LnkSta   │ NCCL maxBw │ Peak allreduce busbw │
├──────────────────────┼───────────┼────────────┼────────────┼──────────────────────┤
│ xio3130 + PF         │ PIX ✅    │ 2.5GT/s ❌ │ 30 GB/s    │ 104 GB/s             │
├──────────────────────┼───────────┼────────────┼────────────┼──────────────────────┤
│ pcie-root-port + PF  │ PHB ❌    │ 32GT/s ✅  │ 48 GB/s    │ 104 GB/s             │
├──────────────────────┼───────────┼────────────┼────────────┼──────────────────────┤
│ Ideal (PIX + 32GT/s) │ PIX ✅    │ 32GT/s ✅  │ ~50 GB/s   │ ~375 GB/s            │
└──────────────────────┴───────────┴────────────┴────────────┴──────────────────────┘

Measured on a 2-node cluster with 16× H100 + 8× NDR 400G NICs per node:
- Single-node NVLink allreduce: 375 GB/s ✅
- Inter-node allreduce (current QEMU): 104 GB/s (~28% of theoretical)
- Inter-node allreduce (with proposed fix): ~375 GB/s (theoretical)

This affects every virtualized GPU cluster using QEMU/KVM with VFIO passthrough for AI/ML training, where NCCL collective bandwidth is the primary performance metric.

Requested Fix

Add x-speed and x-width device properties to xio3130-downstream (and x3130-upstream) following the PCIESlot pattern used by pcie-root-port:

/* hw/pci-bridge/xio3130_downstream.c */
static Property xio3130_downstream_props[] = {
    DEFINE_PROP_UINT8("x-speed", XIO3130Downstream, speed,
                      QEMU_PCI_EXP_LNK_2_5GT),
    DEFINE_PROP_UINT8("x-width", XIO3130Downstream, width,
                      QEMU_PCI_EXP_LNK_X1),
    DEFINE_PROP_END_OF_LIST()
};

This enables -global xio3130-downstream.x-speed=32 -global xio3130-downstream.x-width=16, allowing guest OS and NCCL to correctly see both PIX topology AND Gen5 x16 link speed — critical for AI/ML training on
virtualized GPU clusters.

No Viable Workaround

NCCL_TOPO_FILE can partially override NCCL's link speed estimate but does not fix the PHB vs PIX topology classification or restore correct GPU→NIC DMA path efficiency. Inter-node bandwidth remains at ~28% of
theoretical maximum.