linux-user: substantial memory leak when threads are created and destroyed
Host environment
- Operating system: Fedora 35 Workstation, Windows 11 21H2 22000.469 WSL2
- OS/kernel version: 5.15.14-200.fc35.x86_64, 5.10.60.1-microsoft-standard-WSL2
- Architecture: x86_64
- QEMU flavor: qemu-arm (arm-linux-user), qemu-x86_64 (x86_64-linux-user)
- QEMU version: 6.1.0-10.fc35, and on master@0a301624c2f4ced3331ffd5bce85b4274fe132af
- QEMU command line:
bin/debug/native/qemu-arm -L /arm-none-linux-gnueabihf/arm-none-linux-gnueabihf/libc /mnt/f/test_qemu3.out
Emulated/Virtualized environment
- Operating system: QEMU Linux User Mode Emulation
- Toolchain: gcc-arm-10.2-2020.11-x86_64-arm-none-linux-gnueabihf
- Architecture: ARM
Also reproduces on docker images:arm32v7/ubuntu
,arm32v7/debian
,arm32v7/alpine
.
Description of problem
Substantial memory leak when the following simple program is executed on qemu-arm
,
// compile with `arm-none-linux-gnueabihf-gcc test_qemu.c -o test_qemu.out -pthread`
#include <assert.h>
#include <pthread.h>
#define MAGIC_RETURN ((void *)42)
void *thread_main(void *arg)
{
return MAGIC_RETURN;
}
int main(int argc, char *argv[])
{
size_t i;
for (i = 0;; i++)
{
pthread_t thread;
assert(pthread_create(&thread, NULL, thread_main, NULL) == 0);
void *ret;
assert(pthread_join(thread, &ret) == 0);
assert(ret == MAGIC_RETURN);
}
return 0;
}
Steps to reproduce
export TOOLCHAIN_PREFIX=arm-none-linux-gnueabihf
export ARMSDK=/${TOOLCHAIN_PREFIX}
export SYSROOT=${ARMSDK}/${TOOLCHAIN_PREFIX}/libc
export CC=${ARMSDK}/bin/${TOOLCHAIN_PREFIX}-gcc
- Download the arm toolchain:
curl --output ${TOOLCHAIN_PREFIX}.tar.xz -L 'https://developer.arm.com/-/media/Files/downloads/gnu-a/10.2-2020.11/binrel/gcc-arm-10.2-2020.11-x86_64-arm-none-linux-gnueabihf.tar.xz?revision=d0b90559-3960-4e4b-9297-7ddbc3e52783&la=en&hash=985078B758BC782BC338DB947347107FBCF8EF6B'
mkdir -p ${ARMSDK} && tar xf ${TOOLCHAIN_PREFIX}.tar.xz -C ${ARMSDK} --strip-components=1
$CC test_qemu.c -o test_qemu.out -pthread
qemu-arm -L $SYSROOT ./test_qemu.out
- Observe memory usage keeps ramping up and crashes the process once out of memory.
Additional information
Valgrind annotation logs annot.log generated by a local build on master@0a301624c2f4ced3331ffd5bce85b4274fe132af from
valgrind --xtree-memory=full --xtree-memory-file=xtmemory.kcg bin/debug/native/qemu-arm -L $SYSROOT /mnt/f/test_qemu3.out
# Send CTRL-C before the process crashes due to oom
callgrind_annotate --auto=yes --inclusive=yes --sort=curB:100,curBk:100,totB:100,totBk:100,totFdB:100,totFdBk:100 xtmemory.kcg > annot.log
Suspected cause and resolution
Setting "realized" to false at linux-user/syscall.c:8587 (case TARGET_NR_exit; object_property_set_bool)
on thread exit will not cleanup cpu
's parent reference. cpu
was parented by hw/core/qdev.c:515
when we call cpu_copy
at linux-user/syscall.c:do_fork
(cpu_copy
calls hw/core/cpu-common.c:cpu_create
which then realizes the device with a NULL
BusState*
).
The reference count of OBJECT(cpu)
goes to 1
after the call to object_unref@linux-user/syscall.c:8589
.
Suspecting to be a global linux-user
issue.
Valgrind will not mark cpu
and its resources as lost since cpu
is still referenced by its parent.
A few additional leaks from gdb_stub
and (ARMCPU*)cpu->cpreg_*
can then be detected by valgrind after patching this issue with an additional call to object_unparent(OBJECT(CPU))
prior to object_unref(OBJECT(cpu))
.