vhost_user: Add used vring watchdog

We know that notifications we send to the guest via our call FD can in rare cases be lost before they turn up as an interrupt in the guest kernel’s virtio-fs module. We don’t know the root cause for this, and we don’t know how to properly fix it.

This patch adds a work-around in form of a watchdog: After returning descriptors to the guest via the used vring, we arm a watchdog that checks whether the guest is making progress on processing the used vring. If it is found to be stalled, we re-issue the notification.


My original plan was to implement this in a vhost crate, but then I found this impractical; we should only arm the watchdog when we stop polling a virtqueue, and this is a concept that does not exist in the vhost or vm-virtio crates. Therefore, I decided to implement it purely in virtiofsd after all.

There is a single small layering violation: The vm-virtio/virtio-queue crate does not provide a method to get the used_event value. We can just copy its code for reading it directly from memory (and this is what I’ve done here), but it might be nicer to have such a method in QueueT so we don’t need to get our hands dirty.


Tested with modified kernels that simply ignore interrupts once in a while.

Summary by Sourcery

Implement a watchdog mechanism to detect and handle rare cases of lost interrupts in vhost_user virtqueue processing

New Features:

  • Add a watchdog mechanism to monitor guest processing of used virtual rings
  • Implement a mechanism to re-send notifications if the guest appears to be stuck processing descriptors

Enhancements:

  • Create a robust mechanism to detect interrupt loss in virtqueue processing
  • Provide a configurable timeout for monitoring guest progress

Chores:

  • Add comprehensive comments explaining the watchdog mechanism
  • Implement thread-safe monitoring of virtqueue state
Edited by sourcery-ai

Merge request reports

Loading
Loading