qemu-system-arm regression: NonSecure World can change Secure World MMU mapping.

Host environment

• Operating system: Windows, Linux
• OS/kernel version: Windows 10, Linux 6.10.10
• Architecture: Guest:x86, Host:ARM
• QEMU flavor: qemu-system-arm
• QEMU version: 9.1.0
• QEMU command line: qemu-system-arm -machine mcimx6ul-evk -cpu cortex-a7 -m 512M -semihosting

Emulated/Virtualized environment

• Operating system: ARM Bare Metal (TamaGo)
• Architecture: ARM (ARMv7)

Description of problem

A NonSecure execution context is able to override MMU L1 translation table flags set by Secure context on Secure World memory.

This is not consistent with the same code running on real hardware and it's a regression over past qemu releases as 9.0.0 behaves correctly.

Steps to reproduce

This has been tested with GoTEE-example as follows:

# building tamago
wget https://github.com/usbarmory/tamago-go/archive/refs/tags/latest.zip
unzip latest.zip
cd tamago-go-latest/src && ./all.bash
cd ../bin && export TAMAGO=`pwd`/go

# building and running GoTEE-example
wget https://github.com/usbarmory/GoTEE-example/archive/refs/heads/master.zip
unzip master.zip
cd GoTEE-example
export TARGET=usbarmory && make clean && make nonsecure_os_go && make trusted_applet_go && make trusted_os && make qemu

Actual result (qemu 9.1.0)

qemu-system-arm -machine mcimx6ul-evk -cpu cortex-a7 -m 512M -nographic -monitor none -serial null -serial stdio -net none -semihosting -kernel bin/trusted_os_usbarmory.elf
tamago/arm (go1.23.1) • TEE security monitor (Secure World system/monitor)
...
> gotee
00:00:03 SM loaded applet addr:0x10000000 entry:0x100824fc size:5010875
00:00:03 SM loaded kernel addr:0x80000000 entry:0x800818e8 size:4475501
00:00:03 SM waiting for applet and kernel
00:00:03 SM starting mode:USR sp:0x12000000 pc:0x100824fc ns:false
00:00:03 SM starting mode:SYS sp:0x00000000 pc:0x800818e8 ns:true
00:00:03    r0:00000001  r1:00000030  r2:00000030  r3:80476050
00:00:03    r4:00000047  r5:804320c0  r6:00000047  r7:00000000
00:00:03    r8:80476050  r9:00000001 r10:8040a128 r11:8030668c cpsr:200001d7 (ABT)
00:00:03   r12:00000074  sp:8043acd4  lr:8017d1ac  pc:9800000c spsr:200001d6 (MON)
00:00:03 SM stopped mode:SYS sp:0x8043acd4 lr:0x8017d1ac pc:0x9800000c ns:true err:ABT
00:00:03        :0
00:00:03        :0
...

Expected result (qemu 9.0.0)

qemu-system-arm -machine mcimx6ul-evk -cpu cortex-a7 -m 512M -nographic -monitor none -serial null -serial stdio -net none -semihosting -kernel bin/trusted_os_usbarmory.elf
tamago/arm (go1.23.1) • TEE security monitor (Secure World system/monitor)
...
> gotee
00:00:02 SM loaded applet addr:0x10000000 entry:0x100824fc size:5010875
00:00:02 SM loaded kernel addr:0x80000000 entry:0x800818e8 size:4476055
00:00:02 SM waiting for applet and kernel
00:00:02 SM starting mode:USR sp:0x12000000 pc:0x100824fc ns:false
00:00:02 SM starting mode:SYS sp:0x00000000 pc:0x800818e8 ns:true
00:00:02 tamago/arm (go1.23.1) • TEE user applet
00:00:02 tamago/arm (go1.23.1) • system/supervisor (Non-secure:true)
00:00:02 supervisor is about to yield back
00:00:02 SM stopped mode:SYS sp:0x8043aee8 lr:0x8017cf44 pc:0x8017d360 ns:true err:<nil>
...

Additional information

The issue relates to the fact that the NonSecure World, at startup, configures the MMU with the NX bit for the entire address space not belonging to its firmware .text area.

On real hardware this MMU configuration by NonSecure world does not affect the Secure World translation tables.

On qemu 9.1.0, however it does and this is inconsistent with real hardware behavior. On qemu 9.0.0 the behaviour is correct so the issue has been introduced between these two releases.

The switch between Secure and NonSecure is done here.

The MMU first level address table which sets the NX bit is done here.

Possibly related issues

• #2326 (closed)
• #2568