<!--This is the upstream QEMU issue tracker. If you are able to, it will greatly facilitate bug triage if you attempt to reproduce the problem with the latest qemu.git master built from source. See https://www.qemu.org/download/#source for instructions on how to do this. QEMU generally supports the last two releases advertised on https://www.qemu.org/. Problems with distro-packaged versions of QEMU older than this should be reported to the distribution instead. See https://www.qemu.org/contribute/report-a-bug/ for additional guidance. If this is a security issue, please consult https://www.qemu.org/contribute/security-process/--> ## Host environment - Operating system: Linux x86_64 - OS/kernel version: Linux centos7 6.6.87.2-microsoft-standard-WSL2 - Architecture: x86 - QEMU flavor: qemu-system-riscv64 - QEMU version: v11.0.0-1713-gde5d8bfd61\`, reported version 11.0.50 - QEMU command line: <!--Give the smallest, complete command line that exhibits the problem. </li> </ul> <p data-sourcepos="39:1-40:62">If you are using libvirt, virsh, or vmm, you can likely find the QEMU command line arguments in /var/log/libvirt/qemu/$GUEST.log.--&gt;</p> <pre data-sourcepos="42:3-51:5"><code>qemu-system-riscv64 \ -M virt,iommu-sys=on,aia=aplic-imsic \ -cpu rv64,smstateen=true \ -m 8G \ -trace riscv_iommu_msi \ -nographic \ -device edu,dma_mask=0xFFFFFFFFFFFFFFFF \ -bios bin/iommu_mrif_pending_offset.elf </code></pre> <h2 id="user-content-emulatedvirtualized-environment" data-sourcepos="53:1-53:35">Emulated/Virtualized environment<a href="#emulatedvirtualized-environment" aria-label="Link to heading 'Emulated/Virtualized environment'" data-heading-content="Emulated/Virtualized environment" class="anchor"></a></h2> <ul data-sourcepos="55:1-63:28"> <li data-sourcepos="55:1-57:41"> <p data-sourcepos="55:3-55:19">Operating system:</p> <!--Windows 10 21H1, Fedora 37, etc.--> - OS/kernel version: <!--For POSIX guests, use `uname -a`.--> - Architecture: <!--x86, ARM, s390x, etc.--> ## Description of problem In MRIF (memory-resident interrupt file) mode, the IOMMU computes the address of the interrupt-pending doubleword as `(identity >> 6) * 8`, but the MRIF format places the pending doubleword of group `i = identity >> 6` at byte offset `i * 16` (each group is a 16-byte pending/enable pair). For any interrupt identity \>= 64 the pending bit is therefore written into the wrong doubleword (for identity 64 it lands in group 0's _enable_ doubleword), and the correct pending bit is never set. The memory-resident interrupt file format is defined by the AIA specification (Section 8.3.1) and referenced by the IOMMU specification (Section 3.3.3). ## Steps to reproduce 1. [iommu_mrif_pending_offset.c](/uploads/06c3a045fb40bc211a00cae2e0f48dbc/iommu_mrif_pending_offset.c) 2. Built with: `riscv64-unknown-elf-gcc -march=rv64gc -mabi=lp64 -mcmodel=medany -nostartfiles -nostdlib` 3. [iommu_mrif_pending_offset.elf](/uploads/c578051b9ffa200f1de7e4e27c1b998d/iommu_mrif_pending_offset.elf) 4. qemu-system-riscv64 \ -M virt,iommu-sys=on,aia=aplic-imsic \ -cpu rv64,smstateen=true \ -m 8G \ -trace riscv_iommu_msi \ -nographic \ -device edu,dma_mask=0xFFFFFFFFFFFFFFFF \ -bios bin/iommu_mrif_pending_offset.elf ## Additional information The guest program (`tests/iommu_mrif_pending_offset.c`): 1. Registers a DC with MSI Flat mode and an MRIF-mode MSI PTE whose MRIF address points at a 512-byte buffer `mrif_buf`. 2. Maps gva_msi to the MSI guest physical page; programs the EDU device MSI message data to interrupt identity 64 (\>= 64) and triggers the MSI. 3. Reads `mrif_buf` to see which doubleword had its pending bit set. ## Expected behaviour For interrupt identity I, the pending doubleword is at offset `16 * (I >> 6)` and the bit within is `I & 63`. For identity 64 that is offset `0x10`, bit 0. Spec references: - RISC-V Advanced Interrupt Architecture (AIA) Specification, Section 8.3.1 (Format of a memory-resident interrupt file): "In general, the pair of doublewords at address offsets 16_i and 16_i+8 for integer i contain the interrupt-pending and interrupt-enable bits for external interrupt minor identities in the range 64_i to 64_i+63. For identity in this range, bit (identity mod 64) of the first (even) doubleword is the interrupt-pending bit, and the same bit of the second (odd) doubleword is the interrupt-enable bit." - RISC-V IOMMU Architecture Specification, Section 3.3.3: "In MRIF mode, the Advanced Interrupt Architecture Specification defines the operation to store the incoming MSIs into the destination MRIF and to generate the notice MSI." ## Observed behaviour ``` MSI identity = 0x00000040 to correct pending offset 0x00000010 bit 0x00000000 Result MRIF[correct 0x00000010] = 0x0000000000000000 MRIF[buggy 0x00000008] = 0x0000000000000001 BUG: pending bit set at offset 0x00000008 instead of 0x00000010; the correct pending bit is never set. ``` `riscv_iommu_msi` trace (write goes to MRIF base + 8 instead of + 16): ``` riscv_iommu_msi (null): translate 0000:01.0 MSI 0x80299000000 -> 0x80098008 ``` The MRIF base is 0x80098000; the pending write lands at 0x80098008 (offset 8), which is group 0's enable doubleword. The proper pending doubleword at offset 0x10 stays 0. ## Root cause In `hw/riscv/riscv-iommu.c`, `riscv_iommu_msi_write()`, MRIF mode: ```c /* MRIF pending bit address */ addr = get_field(pte[0], RISCV_IOMMU_MSI_PTE_MRIF_ADDR) << 9; addr = addr | ((data & 0x7c0) >> 3); ... data = 1ULL << (data & 0x03f); ``` `data & 0x7c0` extracts bits \[10:6\] of the identity, i.e. `(identity >> 6) << 6`. Shifting right by 3 yields `(identity >> 6) * 8`, but the pending doubleword offset must be `(identity >> 6) * 16`. The bit selection `data & 0x03f` (= identity & 63) is correct; only the doubleword offset is wrong. ## Notes - For identities 1-63 (group 0) the offset is 0 either way, so the defect is invisible. It affects every identity \>= 64. <!--The line below ensures that proper tags are added to the issue. Please do not remove it.-->