qemu-arm: Unable to reserve 0xffff0000 bytes of virtual address space at 0x1000 (Success) for use as guest address space (check yourvirtual memory ulimit setting, min_mmap_addr or reserve less using -R option)

Description

qemu-arm is used as part of the openbmc build process https://github.com/openbmc/openbmc/. Lately, the builds have been failing due to an error raised from qemu-arm. For example, in commit 2763316cd.

The default vm.mmap_min_addr setting on my RHEL7.4 machine is 4096=0x1000. Changing this setting exactly to 65536, allows qemu-arm to work. Other values, smaller or larger, cause the error.

Error

qemu-arm: Unable to reserve 0xffff0000 bytes of virtual address space at 0x1000 (Success) for use as guest address space (check yourvirtual memory ulimit setting, min_mmap_addr or reserve less using -R option)

More Details

I am wondering why the code in linux-user/elfload.c raises an error if the address reserved by mmap doesn't exactly match the requested address which is more of a hint. When I run an strace on the command line call (shown below in step 2), I see the mmap syscall succeeds but always returns the address 0x10000=65536 regardless of the vm.mmap_min_addr (requested address) setting.

    /* Reserve the memory on the host. */
    assert(guest_base != 0);
    test = g2h_untagged(0);
    addr = mmap(test, reserved_va, PROT_NONE, flags, -1, 0);
    if (addr == MAP_FAILED || addr != test) {
        error_report("Unable to reserve 0x%lx bytes of virtual address "
                     "space at %p (%s) for use as guest address space (check your"
                     "virtual memory ulimit setting, min_mmap_addr or reserve less "
                     "using -R option)", reserved_va, test, strerror(errno));
        exit(EXIT_FAILURE);
    }

I see the logic || addr != test was changed in the following qemu commit:

commit fb730c8683807d549c4aa8dfc98e3d1d470fd784
Author: Alex Benn<C3><A9>e <alex.bennee@linaro.org>
Date:   Tue Sep 15 14:43:10 2020 +0100

    linux-user: test, don't assert addr != test in pgb_reserved_va
    
    On older kernels which don't implement MAP_FIXED_NOREPLACE the kernel
    may still fail to give us the address we asked for despite having
    already probed the map for a valid hole. Asserting isn't particularly
    useful to the user so let us move the check up and expand the
    error_report a little to give them a fighting chance of working around
    the problem.
    
    Signed-off-by: Alex Benn<C3><A9>e <alex.bennee@linaro.org>
    Reviewed-by: Laurent Vivier <laurent@vivier.eu>
    Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
    Cc: Bug 1895080 <1895080@bugs.launchpad.net>
    Ameliorates: ee94743034
    Message-Id: <20200915134317.11110-2-alex.bennee@linaro.org>

QEMU Release

$ ./tmp/sysroots-components/x86_64/qemu-native/usr/bin/qemu-arm --version
qemu-arm version 6.0.0
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers

Host OS Info

$ lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: RedHatEnterpriseWorkstation
Description:    Red Hat Enterprise Linux Workstation release 7.4 (Maipo)
Release:        7.4
Codename:       Maipo

Steps to reproduce

1. sudo sysctl -w vm.mmap_min_addr=4096
2. ./tmp/sysroots-components/x86_64/qemu-native/usr/bin/qemu-arm -r 3.2.0 -E LD_LIBRARY_PATH=/home/bmcbuild/releases/obmc910/Builds/ob910.2125.20210624a/2763316cd41d2ec47013/build/tmp/work/witherspoon-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/rootfs/usr/lib:/home/bmcbuild/releases/obmc910/Builds/ob910.2125.20210624a/2763316cd41d2ec47013/build/tmp/work/witherspoon-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/rootfs/lib -L /home/bmcbuild/releases/obmc910/Builds/ob910.2125.20210624a/2763316cd41d2ec47013/build/tmp/work/witherspoon-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/rootfs -E FC_DEBUG=1 /home/bmcbuild/releases/obmc910/Builds/ob910.2125.20210624a/2763316cd41d2ec47013/build/tmp/work/witherspoon-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/rootfs/usr/libexec/fc-cache --sysroot=/home/bmcbuild/releases/obmc910/Builds/ob910.2125.20210624a/2763316cd41d2ec47013/build/tmp/work/witherspoon-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/rootfs --system-only -v