qemu-user: g++(cc1plus) crash on stack size unlimited environment

Host environment

  • Operating system: ubuntu 20.03
  • OS/kernel version: Linux localhost 5.15.0-139-generic #149 (closed)~20.04.1-Ubuntu SMP Wed Apr 16 08:29:56 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
  • Architecture: x86_64
  • QEMU flavor: qemu-user (qemu-riscv64)
  • QEMU version: 9.2.4
  • QEMU command line: none (using by binfmt-misc)

Emulated/Virtualized environment

  • Operating system: openEuler 24.03 SP2 (container)
  • OS/kernel version: none (qemu user mode)
  • Architecture: riscv64

Description of problem

g++(cc1plus) guest program stack overflow when build nss

crash detail

g++ -o Linux5.15_riscv64_cc_glibc_PTH_64_OPT.OBJ/pk11_aeskeywrapkwp_unittest.o -c -std=c++11 -O2 -fPIC   -pipe -ffunction-sections -fdata-sections -DHAVE_STRERROR -DLINUX -Dlinux -Wall -Wshadow -Werror -Wsign-compare -DXP_UNIX -DXP_UNIX -DDLL_PREFIX=\"lib\" -DDLL_SUFFIX=\"so\" -UDEBUG -DNDEBUG -D_DEFAULT_SOURCE -D_BSD_SOURCE -D_POSIX_SOURCE -DSDB_MEASURE_USE_TEMP_DIR -D_REENTRANT -UDEBUG -DNDEBUG -D_DEFAULT_SOURCE -D_BSD_SOURCE -D_POSIX_SOURCE -DSDB_MEASURE_USE_TEMP_DIR -D_REENTRANT -DNSS_DISABLE_AVX2 -DNSS_DISABLE_SSE3 -DUSE_UTIL_DIRECTLY -DNO_NSPR_10_SUPPORT -DSSL_DISABLE_DEPRECATED_CIPHER_SUITE_NAMES -I../../gtests/google_test/gtest/include -I../../gtests/common -I../../cpputil -I/usr/include/nspr4 -I../../../dist/Linux5.15_riscv64_cc_glibc_PTH_64_OPT.OBJ/include -I../../../dist/public/nss -I../../../dist/private/nss -I../../../dist/public/nspr -I../../../dist/public/nss -I../../../dist/public/libdbm -I../../../dist/public/gtest -I../../../dist/public/cpputil -O1 -g -grecord-gcc-switches -pipe -fstack-protector-strong -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/generic-hardened-cc1 -fasynchronous-unwind-tables -fstack-clash-protection -std=c++0x pk11_aeskeywrapkwp_unittest.cc
g++: internal compiler error: Segmentation fault signal terminated program cc1plus

While ulimit -s is unlimited, the guest stack size is capped at 8 MB. As a result, a guest program will crash if it uses more than 8 MB of stack space. This behavior is inconsistent with native linux program.

#if !defined(TARGET_DEFAULT_STACK_SIZE)
/* XXX: on x86 MAP_GROWSDOWN only works if ESP <= address + 32, so
   we allocate a bigger stack. Need a better solution, for example
   by remapping the process stack directly at the right place */
#define TARGET_DEFAULT_STACK_SIZE	8 * 1024 * 1024UL
#endif

unsigned long guest_stack_size = TARGET_DEFAULT_STACK_SIZE;

int main(int argc, char **argv, char **envp)
{
    ...
    /* Read the stack limit from the kernel.  If it's "unlimited",
       then we can do little else besides use the default.  */
    {
        struct rlimit lim;
        if (getrlimit(RLIMIT_STACK, &lim) == 0
            && lim.rlim_cur != RLIM_INFINITY
            && lim.rlim_cur == (target_long)lim.rlim_cur
            && lim.rlim_cur > guest_stack_size) {
            guest_stack_size = lim.rlim_cur;
        }
    }
    ...
}

I had try:

  1. build nss on riscv64 machine without qemu-riscv64, g++(cc1plus) work will.
  2. using 'ulimit -s $((16*1024))' on riscv64 bash in qemu-user, but set failed.
  3. using 'ulimit -s $((16*1024))' on native bash, then build nss on qemu-user, g++(cc1plus) work will.
  4. modify TARGET_DEFAULT_STACK_SIZE to 16M and re-run this g++ command, g++(cc1plus) work will.

Steps to reproduce

  1. save the source code to stackoverflow.c
// simulate cc1plus using more than 8MB stack

#include <stdio.h>
#include <stdint.h>

#define PAGE_SIZE 4096
#define ROUNDUP(x, y) (((x) + (y)-1) & ~((y)-1))
#define ROUNDUP_PAGE(x) ROUNDUP(x, PAGE_SIZE)

#define STACK_SIZE (10*1024*1024) // 10 MiB
#define SKIP_SIZE PAGE_SIZE

int main(int argc, char *argv[])
{
    uintptr_t stack_bottom = ROUNDUP_PAGE((uintptr_t)argv[0]);
    uint32_t *write_start = (uint32_t*)(stack_bottom - SKIP_SIZE);
    uint32_t *write_end = (uint32_t*)(stack_bottom - STACK_SIZE);

    for (uint32_t *p = write_start; p >= write_end; p--)
    {
        *p = 0xDEADBEEF;
    }
    return 0;
}
  1. run test program
# `ulimit -s` in qemu-user is not work
# run below command in native bash

riscv64-linux-gnu-gcc -static stackoverflow.c -o stackoverflow

ulimit -s $((10*1024))
./stackoverflow  # exit normal

ulimit -s unlimited
./stackoverflow  # crash here

Additional information

I suggest set large value (16M/32M) to TARGET_DEFAULT_STACK_SIZE can prevent guest stack overflow from complex guest program.

Edited by xfan