overflow condition code determined incorrectly after subtraction on s390x

Host environment

• Operating system: Ubuntu 20.04
• OS/kernel version: Linux 5.11.0-27-generic x86_64
• Architecture: x86_64
• QEMU flavor: qemu-s390x
• QEMU version: 6.1.0
• QEMU command line:

  qemu-s390x a.out

Emulated/Virtualized environment

• Operating system: Linux
• OS/kernel version: none
• Architecture: s390x

Description of problem

Paul Eggert found this bug, just by taking a look at the file qemu/target/s390x/tcg/cc_helper.c.

The following program foo.c

#include <stdio.h>

int overflow_32 (int x, int y)
{
  int sum;
  return __builtin_sub_overflow (x, y, &sum);
}

int overflow_64 (long long x, long long y)
{
  long sum;
  return __builtin_sub_overflow (x, y, &sum);
}

int a1 = 0;
int b1 = -2147483648;
long long a2 = 0L;
long long b2 = -9223372036854775808L;

int main ()
{
  {
    int a = a1;
    int b = b1;
    printf ("a = 0x%x, b = 0x%x\n", a, b);
    printf ("no_overflow = %d\n", ! overflow_32 (a, b));
  }
  {
    long long a = a2;
    long long b = b2;
    printf ("a = 0x%llx, b = 0x%llx\n", a, b);
    printf ("no_overflow = %d\n", ! overflow_64 (a, b));
  }
}

should print

a = 0x0, b = 0x80000000
no_overflow = 0
a = 0x0, b = 0x8000000000000000
no_overflow = 0

However, when compiled as an s390x program and executed through qemu 6.1.0 (Linux user-mode), it prints 'no_overflow = 1' twice.

$ s390x-linux-gnu-gcc-10 --version
s390x-linux-gnu-gcc-10 (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0

$ s390x-linux-gnu-gcc-10 -static foo.c
$ ~/inst-qemu/6.1.0/bin/qemu-s390x a.out
a = 0x0, b = 0x80000000
no_overflow = 1
a = 0x0, b = 0x8000000000000000
no_overflow = 1

$ s390x-linux-gnu-gcc-10 -O2 -static foo.c
$ ~/inst-qemu/6.1.0/bin/qemu-s390x a.out
a = 0x0, b = 0x80000000
no_overflow = 1
a = 0x0, b = 0x8000000000000000
no_overflow = 1

The code generated by 's390x-linux-gnu-gcc-10 -O2' makes use of the 'o' (overflow / ones) condition code:

overflow_64:
        lgr     %r1,%r2    ;; copy a into %r1
        lghi    %r2,0
        sgr     %r1,%r3    ;; subtract b from a
        bnor    %r14       ;; if no overflow, return %r2 = 0
        lghi    %r2,1
        br      %r14       ;; otherwise, return %r2 = 1

The condition code and the overflow bit are defined in the z/Architecture Principles of Operation (POP) http://publibfi.boulder.ibm.com/epubs/pdf/dz9zr011.pdf page 7-5 / 7-6 / 7-388 : "In mathematical terms, signed addition and subtraction produce a fixed-point overflow when the result is outside the range of representation for signed binary integers."

I conclude that the bug is in QEMU: QEMU does not set the overflow condition code correctly.

Steps to reproduce

foo.static.s390x (the result of "s390x-linux-gnu-gcc-10 -static -O2 foo.c -o foo.static.s390x")

1. qemu-s390x foo.static.s390x

Additional information

The attached patch fixes it. 0002-s390x-Fix-determination-of-overflow-condition-code-a.patch