Skip to content

test_malta.py sometimes times out (likely hang) under GitLab CI

Host environment

  • GitLab CI runner for 'functional-system-fedora'

Emulated/Virtualized environment

  • Architecture: mips

Description of problem

In functional-system-fedora job I saw a failure of:

 1/57 qemu:func-thorough+func-mips-thorough+thorough / func-mips-malta                  TIMEOUT        480.11s   killed by signal 15 SIGTERM

Normally in CI this job completes in 120 seconds, so a 480 second timeout strongly suggests a hang

The 'build/tests/functional/mips/test_malta.MaltaMachineConsole.test_wheezy/console.log' showed a soft lockup failure

2025-09-08 17:36:46,426: INIT: version 2.88 booting
2025-09-08 17:36:46,942: [[36minfo[39;49m] Using makefile-style concurrent boot in runlevel S.
2025-09-08 17:36:47,378: findfs: unable to resolve 'UUID=042f1883-e9a5-4801-bb9b-667b5c8e87ea'
2025-09-08 17:36:50,448: [....] Starting the hotplug events dispatcher: udevd[?25l[?1c7[1G[[32m ok [39;49m8[?25h[?0c.
2025-09-08 17:36:52,269: [....] Synthesizing the initial hotplug events...module e1000: dangerous R_MIPS_LO16 REL relocation
2025-09-08 17:37:17,707: BUG: soft lockup - CPU#0 stuck for 22s! [modprobe:208]
2025-09-08 17:37:17,707: Modules linked in:
2025-09-08 17:37:17,707: Cpu 0
2025-09-08 17:37:17,708: $ 0   : 00000000 1000a400 0000003d 87808b00
2025-09-08 17:37:17,708: $ 4   : 87808b00 87808bf0 00000000 00000000
2025-09-08 17:37:17,709: $ 8   : 86862100 86862100 86862100 86862100
2025-09-08 17:37:17,709: $12   : 86862100 00000000 00000001 86862100
2025-09-08 17:37:17,709: $16   : 87808a00 86862100 1000a401 c008fa60
2025-09-08 17:37:17,709: $20   : 86862100 8041d230 00000000 ffff0000
2025-09-08 17:37:17,710: $24   : 00000000 77711470
2025-09-08 17:37:17,710: $28   : 87bb6000 87bb7df8 8041d230 801f7388
2025-09-08 17:37:17,710: Hi    : 00000000
2025-09-08 17:37:17,710: Lo    : 00000000
2025-09-08 17:37:17,711: epc   : 801f7308 kfree+0x104/0x19c
2025-09-08 17:37:17,711: Not tainted
2025-09-08 17:37:17,711: ra    : 801f7388 kfree+0x184/0x19c
2025-09-08 17:37:17,712: Status: 1000a403    KERNEL EXL IE
2025-09-08 17:37:17,712: Cause : 50808000
2025-09-08 17:37:17,712: PrId  : 00019300 (MIPS 24Kc)
2025-09-08 17:37:45,707: BUG: soft lockup - CPU#0 stuck for 22s! [modprobe:208]
2025-09-08 17:37:45,707: Modules linked in:

Hard to be sure if this is flaky CI runners causing stalled execution, or a genuine QEMU or Linux guest bug ?

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information