qemu-arm: Unable to reserve 0xffff0000 bytes of virtual address space at 0x1000 (Success) for use as guest address space (check yourvirtual memory ulimit setting, min_mmap_addr or reserve less using -R option)
Description
qemu-arm is used as part of the openbmc build process https://github.com/openbmc/openbmc/. Lately, the builds have been failing due to an error raised from qemu-arm. For example, in commit 2763316cd.
The default vm.mmap_min_addr
setting on my RHEL7.4 machine is 4096=0x1000. Changing this setting exactly to 65536, allows qemu-arm to work. Other values, smaller or larger, cause the error.
Error
qemu-arm: Unable to reserve 0xffff0000 bytes of virtual address space at 0x1000 (Success) for use as guest address space (check yourvirtual memory ulimit setting, min_mmap_addr or reserve less using -R option)
More Details
I am wondering why the code in linux-user/elfload.c
raises an error if the address reserved by mmap doesn't exactly match the requested address which is more of a hint. When I run an strace on the command line call (shown below in step 2), I see the mmap syscall succeeds but always returns the address 0x10000=65536 regardless of the vm.mmap_min_addr (requested address) setting.
/* Reserve the memory on the host. */
assert(guest_base != 0);
test = g2h_untagged(0);
addr = mmap(test, reserved_va, PROT_NONE, flags, -1, 0);
if (addr == MAP_FAILED || addr != test) {
error_report("Unable to reserve 0x%lx bytes of virtual address "
"space at %p (%s) for use as guest address space (check your"
"virtual memory ulimit setting, min_mmap_addr or reserve less "
"using -R option)", reserved_va, test, strerror(errno));
exit(EXIT_FAILURE);
}
I see the logic || addr != test
was changed in the following qemu commit:
commit fb730c8683807d549c4aa8dfc98e3d1d470fd784
Author: Alex Benn<C3><A9>e <alex.bennee@linaro.org>
Date: Tue Sep 15 14:43:10 2020 +0100
linux-user: test, don't assert addr != test in pgb_reserved_va
On older kernels which don't implement MAP_FIXED_NOREPLACE the kernel
may still fail to give us the address we asked for despite having
already probed the map for a valid hole. Asserting isn't particularly
useful to the user so let us move the check up and expand the
error_report a little to give them a fighting chance of working around
the problem.
Signed-off-by: Alex Benn<C3><A9>e <alex.bennee@linaro.org>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Cc: Bug 1895080 <1895080@bugs.launchpad.net>
Ameliorates: ee94743034
Message-Id: <20200915134317.11110-2-alex.bennee@linaro.org>
QEMU Release
$ ./tmp/sysroots-components/x86_64/qemu-native/usr/bin/qemu-arm --version
qemu-arm version 6.0.0
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
Host OS Info
$ lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: RedHatEnterpriseWorkstation
Description: Red Hat Enterprise Linux Workstation release 7.4 (Maipo)
Release: 7.4
Codename: Maipo
Steps to reproduce
sudo sysctl -w vm.mmap_min_addr=4096
./tmp/sysroots-components/x86_64/qemu-native/usr/bin/qemu-arm -r 3.2.0 -E LD_LIBRARY_PATH=/home/bmcbuild/releases/obmc910/Builds/ob910.2125.20210624a/2763316cd41d2ec47013/build/tmp/work/witherspoon-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/rootfs/usr/lib:/home/bmcbuild/releases/obmc910/Builds/ob910.2125.20210624a/2763316cd41d2ec47013/build/tmp/work/witherspoon-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/rootfs/lib -L /home/bmcbuild/releases/obmc910/Builds/ob910.2125.20210624a/2763316cd41d2ec47013/build/tmp/work/witherspoon-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/rootfs -E FC_DEBUG=1 /home/bmcbuild/releases/obmc910/Builds/ob910.2125.20210624a/2763316cd41d2ec47013/build/tmp/work/witherspoon-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/rootfs/usr/libexec/fc-cache --sysroot=/home/bmcbuild/releases/obmc910/Builds/ob910.2125.20210624a/2763316cd41d2ec47013/build/tmp/work/witherspoon-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/rootfs --system-only -v