Skip to content

The GPIO controllers connected to the emulated PCIe bus via vhost-user can't generate interrupts.

Host environment

  • Operating system: Debian/testing Linux
  • OS/kernel version: Linux wzab 6.6.13-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.6.13-1 (2024-01-20) x86_64 GNU/Linux
  • Architecture: x86
  • QEMU flavor: qemu-aarch64
  • QEMU version: 8.1.1
  • QEMU command line:
./qemu-system-aarch64 -M virt -m 1G -cpu cortex-a53 -nographic -chardev socket,path=/tmp/gpio.sock0,id=vgpio -device vhost-user-gpio-pci,chardev=vgpio,id=gpio -object memory-backend-file,id=mem,size=1G,mem-path=/dev/shm,share=on -numa node,memdev=mem  
-smp 1 -kernel Image -append "rootwait root=/dev/vda console=ttyAMA0" -drive file=rootfs.ext4,if=none,format=raw,id=hd0 -device virtio-blk-device,drive=hd0

Emulated/Virtualized environment

  • Operating system: Linux (built with Buildroot,based on qemu_aarch64_virt_defconfig)

  • OS/kernel version: Linux buildroot 6.1.44 #1 SMP Sat Feb 3 15:08:27 CET 2024 aarch64 GNU/Linux

  • Architecture: Aarch64

Description of problem

The problem is related to emulation of GPIO controllers using the vhost-user protocol for GPIO. The problem was detected when using the vhost-device-gpio software. I have described the whole issue in https://github.com/rust-vmm/vhost-device/issues/613 , but it is QEMU related, and therefore I describe it here as well. The broader context is described in https://stackoverflow.com/questions/75906208/how-to-connect-via-virtio-gui-running-on-host-with-gpio-in-a-qemu-emulated-virtu .

Steps to reproduce

  1. For Debian/testing you need to compile a libgpiod-2.1.1 (I assume that the following is done in the home directory directory of the dev user: /home/dev):

    wget https://git.kernel.org/pub/scm/libs/libgpiod/libgpiod.git/snapshot/libgpiod-2.1.tar.gz ; \
    tar -xzf libgpiod-2.1.tar.gz ; \
    cd libgpiod-2.1 ; \
    autoupdate ; \
    ./autogen.sh ; \
    make
  2. Download the vhost-device-gpio (git clone https://github.com/rust-vmm/vhost-device.git)

  3. Build the vhost-device-gpio (in the vhost-device-gpio subdirectory)

    export PATH_TO_LIBGPIOD=/home/dev/libgpiod-2.1
    export SYSTEM_DEPS_LIBGPIOD_NO_PKG_CONFIG=1
    export SYSTEM_DEPS_LIBGPIOD_SEARCH_NATIVE="${PATH_TO_LIBGPIOD}/lib/.libs/"
    export SYSTEM_DEPS_LIBGPIOD_LIB=gpiod
    export SYSTEM_DEPS_LIBGPIOD_INCLUDE="${PATH_TO_LIBGPIOD}/include/"
    cargo build --features "mock_gpio"
  4. Start vhost-device-gpio: (LD_LIBRARY_PATH=/home/emb/libgpiod-2.1/lib/.libs/ ./vhost-device-gpio -s /tmp/gpio.sock -l s4)

  5. Download the Buildroot 2023.11.1 (wget https://buildroot.org/downloads/buildroot-2023.11.1.tar.xz in another directory) and unpack it. Buildroot and the main directory of Buildroot tree are denoted by BR if the following description.

  6. Configure BR (run make qemu_aarch64_virt_defconfig in the main BR directory, run make menuconfig and select external toolchain, BR2_PACKAGE_LIBGPIOD=y, BR2_PACKAGE_LIBGPIOD_TOOLS=y, run make linux-menuconfig and select CONFIG_GPIO_VIRTIO=m in the kernel configuration)

  7. Build the Linux and QEMU (run make in the BR directory).

  8. Run the emulation in BR/output/images, using the command line given above.

  9. After the virtual machine starts, log in as root and load the driver: modprobe gpio-virtio

  10. Try to monitor changes of one of the emulated pins: gpiomon 0 0

  11. You'll get the error message:

    gpiomon: error waiting for events: No such device

Analysis and workaround

The problem is associated with the fact that the VIRTIO_GPIO_F_IRQ feature is not set in host_features for the emulated device.

I have worked it around by modifying the virtio_pci_pre_plugged function:

static void virtio_pci_pre_plugged(DeviceState *d, Error **errp)
{
    VirtIOPCIProxy *proxy = VIRTIO_PCI(d);
    VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);

    if (virtio_pci_modern(proxy)) {
        virtio_add_feature(&vdev->host_features, 0); /* VIRTIO_GPIO_F_IRQ - added by WZab as a quick&dirty workaround */ 
        virtio_add_feature(&vdev->host_features, VIRTIO_F_VERSION_1);
    }

    virtio_add_feature(&vdev->host_features, VIRTIO_F_BAD_FEATURE);
}

Whether this feature should be added based on the virtio_pci_modern(proxy) condition is not clear to me. However, after the above modification, the IRQ emulation started to work correctly.

I attach the patch that I have added to my BR instance (in the global patch directory in qemu subdirectory.

Additional information

0009-enable-F-IRQ-in-virtio-pci.patch

Edited by Wojciech Zabołotny
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information