Skip to content

vfio/nvgrace-gpu: Add vfio pci variant module for grace hopper

Donald Dutile requested to merge ddutile/centos-stream-9:j36542 into main

JIRA: https://issues.redhat.com/browse/RHEL-36542 JIRA: https://issues.redhat.com/browse/RHEL-36543 JIRA: https://issues.redhat.com/browse/RHEL-36544

Adding kernel pieces for VFIO-PCI variant module for NVIDIA's Grace GPU. This module is used for vgpu on Grace-Hopper, and eventually, Grace-Blackwell.

The four commits enable the assignment to a guest. To test, NVIDIA is providing a qemu-kvm invocation that will pass the VFIO-PCI device to the variant driver in the guest.
This is basic vfio-pci device passthrough enablement, and requires further SMMUv3 updates to make the functionality complete and performant.

The four commits backport upstream commits listed in RHEL-36542, -36543, & -36544 which were not included in the kvm: arm: part-1 backport series (by Sebastian Ott).

The (first) push will initially be marked as Draft, until NVIDIA provides a method for RH-QE to test the ability to assign a vfio-pci variant device to a guest. Once that's been verified by this author, this MR's draft status will be removed, and the rest of the kernel workflow can continue.

Pushing this series now so eyes can see it, and others who want to grab it for testing or additional patch additions can do so.

Update: (July 19, 2024) Pulling-out of draft mode: Verified via modified qemu-kvm command line and added instructions in RHEL-19393 (from the non-RHEL qemu-system-aarch64 cmdline), since QE now can test/verify basic assignment functionality.

v2: Remove CONFIG for RHEL; disabling build since testing showed it could enable crash of guest if nvidia-smi test was run after assignment of Hopper vGPU to guest. Upstream commit to repair unlikely to make 9.5 functional freeze. Take incremental step of getting 90% of functionality into RHEL kernel, and await last vma-related, v2 upstream post & acceptance of partner patch. No code changes.

Signed-off-by: Donald Dutile ddutile@redhat.com

Edited by Donald Dutile

Merge request reports