qemu-system-arm: unable to use LPAE

Host environment

• Operating system: Ubuntu 18.04.6 LTS
• OS/kernel version: Linux 4.15.0-163-generic #171 (closed)-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
• Architecture: x86_64
• QEMU flavor: qemu-system-arm
• QEMU version: QEMU emulator version 7.0.50 (v7.0.0-1865-g9ac873a4-dirty)
• QEMU command line: ./qemu-system-arm
  -machine virt,highmem=on
  -cpu cortex-a15 -smp 4
  -m 4096
  -kernel ./zImage
  -drive id=disk0,file=./rootfs.ext4,if=none,format=raw
  -object rng-random,filename=/dev/urandom,id=rng0
  -device virtio-rng-pci,rng=rng0
  -device virtio-blk-device,drive=disk0
  -device virtio-gpu-pci
  -serial mon:stdio -serial null
  -nographic
  -append 'root=/dev/vda rw mem=4096M ip=dhcp console=ttyAMA0 console=hvc0'

Emulated/Virtualized environment

• Operating system: irrelevant
• OS/kernel version: 5.15.38-yocto-standard
• Architecture: qemuarm cortex-a15

Description of problem

Failed to run qemu: qemu-system-arm: Addressing limited to 32 bits, but memory exceeds it by 1073741824 bytes

Steps to reproduce

1. ./configure --target-list=arm-softmmu
2. make

./qemu-system-arm
-machine virt,highmem=on
-cpu cortex-a15 -smp 4
-m 4096
-kernel ./zImage
-drive id=disk0,file=./rootfs.ext4,if=none,format=raw
-object rng-random,filename=/dev/urandom,id=rng0
-device virtio-rng-pci,rng=rng0
-device virtio-blk-device,drive=disk0
-device virtio-gpu-pci
-serial mon:stdio -serial null
-nographic
-append 'root=/dev/vda rw mem=4096M ip=dhcp console=ttyAMA0 console=hvc0'

Additional information

We set physical address bits to 40 if ARM_FEATURE_LPAE is enabled. But ARM_FEATURE_V7VE also implies ARM_FEATURE_LPAE as set later in arm_cpu_realizefn.

We should add condition for ARM_FEATURE_V7VE, otherwise we would not be able to use highmem larger than 3GB even though we have enabled highmem, since we would fail and return right from machvirt_init.

I have already made a patch to fix this issue. realhezhe/qemu@4dad8167 hw/arm/virt.c

        if (object_property_get_bool(cpuobj, "aarch64", NULL)) {
            pa_bits = arm_pamax(armcpu);
        } else if (arm_feature(&armcpu->env, ARM_FEATURE_LPAE)) {
        } else if (arm_feature(&armcpu->env, ARM_FEATURE_LPAE)
                || arm_feature(&armcpu->env, ARM_FEATURE_V7VE)) {
            /* v7 with LPAE */
            pa_bits = 40;
        } else {

After applying the patch, I can make sure that the pa_bits has already been set to 40, but qemu hangs later. By bisecting I found if the following commit is reverted qemu can boot up successfully.. 39a1fd25 ("target/arm: Fix handling of LPAE block descriptors")

It can't be quickly determined what's going on here at my side. Maybe the author can help give some hints. Thanks.