virtiofs mount hangs in guest after Fedora 39 upgrade
After upgrading to Fedora 39 yesterday (which switched virtiofsd from the C implementation to the Rust implementation), some of my virtiofs mounts have started to hang inside the guest. When this happens, any process in the guest VM that tries to access the mountpoint gets stuck in the sleep state. If I try to `umount -l <mountpoint> && mount <mountpoint>`, then the `mount` command similarly hangs forever. The hang doesn't occur immediately after the guest boots. It'll work fine for a little while and then suddenly hang later. In my test VM, I've seen this happen on a virtiofs mountpoint that has no files open and is just idling. Package versions: * Both host and guest run Fedora 39 (x86_64) * `kernel-6.5.8-300.fc39.x86_64` * `libvirt-daemon-9.7.0-1.fc39.x86_64` * `virtiofsd-1.7.0-5.fc39.x86_64` Inside the guest, when the hang occurs, I don't see any logs at all--neither in `dmesg` nor `journalctl`. On the host, `/var/log/libvirt/qemu/svc-1-fs0-virtiofsd.log` contains: <details> <summary>(click to expand)</summary> ``` virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 9 fv_queue_thread: Start for queue 1 kick_fd 12 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 9 fv_queue_thread: Start for queue 1 kick_fd 12 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 12 fv_queue_thread: Start for queue 0 kick_fd 9 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 9 fv_queue_thread: Start for queue 1 kick_fd 12 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=9 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=12 fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 9 fv_queue_thread: Start for queue 1 kick_fd 12 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=9 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=12 fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 9 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 12 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 13 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_thread: Start for queue 0 kick_fd 10 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 virtio_loop: ppoll interrupted, going around stop_all_queues: Stopping queue 0 thread fv_queue_thread: kill event on queue 0 - quitting stop_all_queues: Stopping queue 1 thread fv_queue_thread: kill event on queue 1 - quitting virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_thread: Start for queue 0 kick_fd 10 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 virtio_loop: ppoll interrupted, going around stop_all_queues: Stopping queue 0 thread fv_queue_thread: kill event on queue 0 - quitting stop_all_queues: Stopping queue 1 thread fv_queue_thread: kill event on queue 1 - quitting virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=10 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=13 fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 13 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 13 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 10 fv_queue_thread: Start for queue 1 kick_fd 13 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_thread: Start for queue 1 kick_fd 15 virtio_loop: ppoll interrupted, going around stop_all_queues: Stopping queue 0 thread fv_queue_thread: kill event on queue 0 - quitting stop_all_queues: Stopping queue 1 thread fv_queue_thread: kill event on queue 1 - quitting virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 virtio_loop: ppoll interrupted, going around stop_all_queues: Stopping queue 0 thread fv_queue_thread: kill event on queue 0 - quitting stop_all_queues: Stopping queue 1 thread fv_queue_thread: kill event on queue 1 - quitting virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_thread: Start for queue 1 kick_fd 15 virtio_loop: ppoll interrupted, going around stop_all_queues: Stopping queue 0 thread fv_queue_thread: kill event on queue 0 - quitting stop_all_queues: Stopping queue 1 thread fv_queue_thread: kill event on queue 1 - quitting virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_thread: Start for queue 1 kick_fd 15 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 virtio_loop: Unexpected poll revents 11 virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_thread: Start for queue 0 kick_fd 11 virtio_loop: ppoll interrupted, going around stop_all_queues: Stopping queue 0 thread fv_queue_thread: kill event on queue 0 - quitting stop_all_queues: Stopping queue 1 thread fv_queue_thread: kill event on queue 1 - quitting virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 virtio_loop: ppoll interrupted, going around stop_all_queues: Stopping queue 0 thread fv_queue_thread: kill event on queue 0 - quitting stop_all_queues: Stopping queue 1 thread fv_queue_thread: kill event on queue 1 - quitting virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_thread: Start for queue 1 kick_fd 15 virtio_loop: ppoll interrupted, going around stop_all_queues: Stopping queue 0 thread fv_queue_thread: kill event on queue 0 - quitting stop_all_queues: Stopping queue 1 thread fv_queue_thread: kill event on queue 1 - quitting virtio_loop: Exit virtio_session_mount: Waiting for vhost-user socket connection... virtio_session_mount: Received vhost-user socket connection virtio_loop: Entry fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 fv_queue_set_started: qidx=0 started=0 fv_queue_thread: kill event on queue 0 - quitting fv_remove_watch: TODO! fd=11 fv_queue_set_started: qidx=1 started=0 fv_queue_thread: kill event on queue 1 - quitting fv_remove_watch: TODO! fd=15 fv_queue_set_started: qidx=0 started=1 fv_queue_thread: Start for queue 0 kick_fd 11 fv_queue_set_started: qidx=1 started=1 fv_queue_thread: Start for queue 1 kick_fd 15 virtio_loop: ppoll interrupted, going around stop_all_queues: Stopping queue 0 thread fv_queue_thread: kill event on queue 0 - quitting stop_all_queues: Stopping queue 1 thread fv_queue_thread: kill event on queue 1 - quitting virtio_loop: Exit [2023-10-24T23:53:36Z WARN virtiofsd] Use of deprecated option format '-o': Please specify options without it (e.g., '--cache auto' instead of '-o cache=auto') [2023-10-24T23:53:36Z INFO virtiofsd] Waiting for vhost-user socket connection... [2023-10-24T23:53:36Z INFO virtiofsd] Client connected, servicing requests ``` </details> The virtiofsd command is launched by libvirt + qemu: ``` root 4537 0.0 0.0 6168 3840 ? S Oct24 0:00 /usr/libexec/virtiofsd --fd=46 -o source=/satapool0/enc/public root 4541 0.0 0.0 8513096 10896 ? Sl Oct24 0:02 /usr/libexec/virtiofsd --fd=46 -o source=/satapool0/enc/public ``` and the volume is mounted in the guest's fstab with: ``` public /media/public virtiofs ro,context=system_u:object_r:httpd_sys_content_t:s0 0 0 ``` The libvirt XML for virtiofs contains: ```xml <filesystem type='mount' accessmode='passthrough'> <driver type='virtiofs'/> <binary path='/usr/libexec/virtiofsd'/> <source dir='/satapool0/enc/public'/> <target dir='public'/> <alias name='fs0'/> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </filesystem> ``` Given the lack of relevant log messages (at least that I can see), I'm not sure how to troubleshoot this. I'm happy to test anything or grab additional information if needed. I currently have the VM left in the hanging state.
issue