Illegal instruction in memset under qemu-user for riscv64

Host environment

• Operating system: Ubuntu 22.04, 24.04, 25.10
• OS/kernel version: default
• Architecture: x86_64
• QEMU flavor: qemu-user
• QEMU version: qemu-user_10.1.0+ds-5ubuntu3 (from Ubuntu Launchpad); also tested with Debian qemu-user packages (e.g. qemu-user_10.2.0~rc1+ds-1, qemu-user_10.0.6+ds-0+deb13u2)，and latest code (v10.2.0-rc1-78-g9ef49528b5)
• QEMU command line / registration: The riscv64 qemu-user static binary was copied to the host and registered via binfmt_misc instead of being invoked with a long qemu command line. Example steps used on the host:

  sudo cp qemu-user_10.1.0+ds-5ubuntu3_amd64/usr/bin/qemu-riscv64 /usr/bin/qemu-riscv64-static
  
  echo -1 > /proc/sys/fs/binfmt_misc/qemu-riscv64
  echo ':qemu-riscv64:M:0:\x7f\x45\x4c\x46\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-riscv64-static:POCF' >/proc/sys/fs/binfmt_misc/register

Emulated/Virtualized environment

• Operating system: Ubuntu 26.04 riscv64 container image
• OS/kernel version: container uses host kernel
• Architecture: riscv64 (RISC-V 64-bit)

Description of problem

While running cmake (and other build steps) inside a linux/riscv64 Ubuntu 26.04 container on an x86_64 host using qemu-user (qemu-riscv64-static) registered via binfmt_misc, cmake sometimes crashes with "Illegal instruction (core dumped)" or "died with signal 4". The illegal instruction is observed within glibc's memset implementation at an instruction that uses the RISC-V vector extension (vse64.v). The failure is intermittent (approximately 50% reproduction rate, with large variance). Using a scalar-only memset via LD_PRELOAD or running under gdb / enabling QEMU_STRACE significantly reduces or eliminates the failure. These observations strongly suggest a bug in qemu-user's vector handling / code generation / state management rather than a cmake bug.

Steps to reproduce

1. On an x86_64 host, fetch qemu-user .deb and extract the qemu-riscv64 binary:

   wget https://launchpad.net/ubuntu/+source/qemu/1:10.1.0+ds-5ubuntu3/+build/31393935/+files/qemu-user_10.1.0+ds-5ubuntu3_amd64.deb
   dpkg-deb -x qemu-user_10.1.0+ds-5ubuntu3_amd64.deb qemu-user_10.1.0+ds-5ubuntu3_amd64
   sudo cp qemu-user_10.1.0+ds-5ubuntu3_amd64/usr/bin/qemu-riscv64 /usr/bin/qemu-riscv64

2. Register qemu-riscv64 with binfmt_misc:

   echo -1 > /proc/sys/fs/binfmt_misc/qemu-riscv64
   echo ':qemu-riscv64:M:0:\x7f\x45\x4c\x46\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-riscv64:POCF' >/proc/sys/fs/binfmt_misc/register

3. Start a riscv64 Ubuntu 26.04 container and enter it:

   docker run --platform=linux/riscv64 --name ubuntu26 -itd ubuntu:26.04 bash
   docker exec -it ubuntu26 bash -i

4. Inside the riscv64 container:

   apt update
   apt install -y build-essential cmake

5. Reproducer A: query system information for cmake

   cmake --system-information

   -> Often fails with:

   bash: [15: 1 (255)] tcsetattr: Inappropriate ioctl for device
   Illegal instruction (core dumped)

6. Reproducer B: minimal CMake project

   • Create files: CMakeLists.txt:

     cmake_minimum_required(VERSION 3.10)
     project(HelloCMake C)
     add_executable(hello main.c)

     main.c:

     #include <stdio.h>
     int main() {
         printf("Hello, CMake!\n");
         return 0;
     }

   • Run:

     mkdir test_cmake
     # copy the two files into test_cmake/
     cd test_cmake
     cmake .

   -> Crash observed during "Detecting C compiler ABI info":

   -- Detecting C compiler ABI info
   bash: line 1:  8489 Illegal instruction        (core dumped) cmake .

7. Reproducer C: bootstrapping/building cmake from source inside container

   apt source cmake
   cd cmake
   apt-get build-dep .
   dpkg-buildpackage -us -uc -b

   -> Bootstrapping errors with:

   Illegal instruction (core dumped)
   Error when bootstrapping CMake:
   Problem while running initial CMake

Observed crash location / debugging data

• From gdb/QEMU_STRACE when available, the illegal instruction is inside glibc's memset implementation:
  • Symbol/location: memset@@GLIBC_2.27+0x52
  • Faulting RISC-V instruction: vse64.v v1,(a5) (vector store of 64-bit elements)
• The crash is intermittent (~50% reproduction rate) and exhibits large variance between runs.
• Behavior when altering execution:
  • LD_PRELOAD a scalar-only memset (libnovecmem.so) — almost completely avoids (99%) the crash.
  • Running the failing process under gdb or setting QEMU_STRACE=1 also makes the crash much less likely.
• The same workload does not reproduce the crash when run under qemu-system (full-system emulation). The issue appears specific to qemu-user.

Workarounds

• LD_PRELOAD a scalar-only memcpy/memset implementation (e.g., libnovecmem.so) to prevent glibc from using vectorized memset.
• Run under gdb (slower) or enable QEMU_STRACE=1 — both reduce the likelihood of triggering the bug.

Originally reported at: https://bugs.launchpad.net/bugs/2133188