target/i386: 16- and 32-bit IDIV of the minimum dividend by -1 crashes QEMU with a host SIGFPE

Host environment

• Operating system: Ubuntu, Linux 7.0, x86-64 (AMD Ryzen 9 9950X)
• QEMU: master at a1c27ca0 (11.1.50), configured with --target-list=x86_64-linux-user,x86_64-softmmu; also 10.2.1 (Debian 1:10.2.1+ds-1ubuntu3.2)
• QEMU flags: none for linux-user (qemu-x86_64 ./idiv 32); for system mode, -accel tcg -machine q35 -m 256M -nographic -no-reboot -kernel bzImage -initrd initramfs.cpio

Emulated CPU

Default (qemu64), TCG.

Description of problem

IDIV r/m16 with DX:AX = 0x8000:0x0000 and IDIV r/m32 with EDX:EAX = 0x80000000:0x00000000, each dividing by -1, should raise #DE, because the quotient (2^15 or 2^31) does not fit the destination. Linux delivers #DE to the guest as SIGFPE with si_code == FPE_INTDIV.

Instead, QEMU itself dies of a host SIGFPE:

qemu-x86_64: QEMU internal SIGFPE {code=INTDIV, addr=0x7aab002b0f0a}
Floating point exception (core dumped)

helper_idivw_AX and helper_idivl_EAX in target/i386/tcg/int_helper.c perform the division on the host before checking whether the quotient fits:

void helper_idivw_AX(CPUX86State *env, target_ulong t0)
{
    int num, den, q, r;

    num = (env->regs[R_EAX] & 0xffff) | ((env->regs[R_EDX] & 0xffff) << 16);
    den = (int16_t)t0;
    ...
    q = (num / den);                  /* INT32_MIN / -1 */
    if (q != (int16_t)q) {
        raise_exception_ra(env, EXCP00_DIVZ, GETPC());
    }

void helper_idivl_EAX(CPUX86State *env, target_ulong t0)
{
    int den, r;
    int64_t num, q;
    ...
    q = (num / den);                  /* INT64_MIN / -1 */
    r = (num % den);
    if (q != (int32_t)q) {
        raise_exception_ra(env, EXCP00_DIVZ, GETPC());
    }

INT32_MIN / -1 and INT64_MIN / -1 are undefined behaviour in C, and on an x86 host they compile to an idiv that traps. On hosts whose division does not trap (AArch64's sdiv returns INT_MIN), the following check happens to raise #DE, so the crash is presumably limited to x86 hosts.

The 8-bit form is not affected (its INT16_MIN / -1 is done in int), and the 64-bit form is not affected (idiv64 checks before it divides); the reproducer shows both raising #DE correctly.

The same helpers serve qemu-system-x86_64 under TCG, so a user-mode (CPL 3) guest process takes the whole VM down. Booting a minimal Linux 7.2.6 guest (-accel tcg -machine q35 -nographic) whose init forks a child for each IDIV form: the 64-bit form raises #DE in the guest as it should; the 16-bit or 32-bit form kills qemu-system-x86_64 with SIGFPE (exit status 136), printing nothing. With -accel kvm all three forms raise #DE in the guest.

The same class of bug was fixed for s390x (commit 0103cb1c) and is reported for m68k in #2078.

Steps to reproduce

1. Save the program below as idiv.c and build it on an x86-64 host: gcc -O2 -static -o idiv idiv.c
2. Run ./idiv 32 natively. It prints guest SIGFPE, FPE_INTDIV: correct and exits 0.
3. Run qemu-x86_64 ./idiv 32 (or ./idiv 16). QEMU prints QEMU internal SIGFPE {code=INTDIV, ...} and is killed by SIGFPE (exit status 136).

./idiv 8 and ./idiv 64 run correctly both natively and under QEMU.

/* IDIV of the minimum dividend by -1 must raise #DE (SIGFPE, FPE_INTDIV).
 * Build: gcc -O2 -static -o idiv idiv.c
 * Run:   ./idiv 8 | 16 | 32 | 64, natively and under qemu-x86_64. */
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

static void on_fpe(int sig, siginfo_t *si, void *uc)
{
    (void)sig, (void)uc;
    static const char ok[] = "guest SIGFPE, FPE_INTDIV: correct\n";
    static const char other[] = "guest SIGFPE, unexpected si_code\n";
    if (si->si_code == FPE_INTDIV)
        _exit(write(1, ok, sizeof ok - 1) < 0);
    _exit(2 + (write(1, other, sizeof other - 1) < 0));
}

int main(int argc, char **argv)
{
    struct sigaction sa;
    memset(&sa, 0, sizeof sa);
    sa.sa_sigaction = on_fpe;
    sa.sa_flags = SA_SIGINFO;
    sigaction(SIGFPE, &sa, 0);

    int w = argc > 1 ? atoi(argv[1]) : 32;
    printf("idiv%d: minimum dividend / -1\n", w);
    fflush(stdout);
    if (w == 8) /* AX = 0x8000 */
        __asm__ volatile("mov $0x8000, %%eax; mov $-1, %%ecx; idiv %%cl"
                         ::: "eax", "ecx", "cc");
    else if (w == 16) /* DX:AX = 0x8000:0000 */
        __asm__ volatile("mov $0x8000, %%edx; xor %%eax, %%eax; mov $-1, %%ecx; idiv %%cx"
                         ::: "eax", "ecx", "edx", "cc");
    else if (w == 32) /* EDX:EAX = 0x80000000:00000000 */
        __asm__ volatile("mov $0x80000000, %%edx; xor %%eax, %%eax; mov $-1, %%ecx; idiv %%ecx"
                         ::: "eax", "ecx", "edx", "cc");
    else /* RDX:RAX = 0x8000000000000000:0 */
        __asm__ volatile("mov $1, %%edx; shl $63, %%rdx; xor %%eax, %%eax; mov $-1, %%rcx; idiv %%rcx"
                         ::: "rax", "rcx", "rdx", "cc");
    puts("no fault: wrong");
    return 1;
}

Additional information

A possible fix: do the 16-bit division in int64_t (sign-extending the 32-bit dividend), where INT32_MIN / -1 is defined, and reject the one overflowing 32-bit case before dividing:

void helper_idivw_AX(CPUX86State *env, target_ulong t0)
{
    int64_t num, q;
    int den, r;

    num = (int32_t)((env->regs[R_EAX] & 0xffff) |
                    ((env->regs[R_EDX] & 0xffff) << 16));
    ...

void helper_idivl_EAX(CPUX86State *env, target_ulong t0)
{
    ...
    if (den == 0 || (num == INT64_MIN && den == -1)) {
        raise_exception_ra(env, EXCP00_DIVZ, GETPC());
    }

With this change on master, the reproducer raises #DE for all three widths in both qemu-x86_64 and qemu-system-x86_64. The two changed helpers were also checked, as standalone C under UBSan, against a 128-bit reference on 20 million dividend/divisor pairs including the minimum dividends and -1, and a randomized comparison of about 850 x86-64 instruction forms (2.5 million tests, all DIV and IDIV forms among them) against a semantics model shows no change. I can send it as a patch if that helps.

Found while validating an x86 semantics model against QEMU, real AMD hardware and Intel SDE; the hardware and SDE both raise #DE here.