Skip to content

The "replay" functional functional test on mips64el is unreliable

Host environment

  • Operating system: Linux
  • OS/kernel version: Ubuntu
  • Architecture: x86_64
  • QEMU flavor: qemu-system-mips64el
  • QEMU version: 8.2rc2
  • QEMU command line:
    ./pyvenv/bin/avocado run ./tests/avocado/replay_kernel.py:ReplayKernelNormal.test_mips64el_malta

Emulated/Virtualized environment

  • OS/kernel version: 2.6.32-5-5kc-malta
  • Architecture: mips64el

Description of problem

This test keeps hanging on CI

Steps to reproduce

Run the test on GitLab's CI infrastructure and it will hang on replay. Examples: https://gitlab.com/stsquad/qemu/-/jobs/5664260736

Additional information

Excerpt from log:

18:02:49 DEBUG| Transitioning from 'Runstate.CONNECTING' to 'Runstate.RUNNING'.
18:02:49 DEBUG| Opening console file
18:02:49 DEBUG| Opening console socket
18:02:49 DEBUG| [    0.000000] Initializing cgroup subsys cpuset
18:02:49 DEBUG| [    0.000000] Initializing cgroup subsys cpu
18:02:49 DEBUG| [    0.000000] Linux version 2.6.32-5-5kc-malta (Debian 2.6.32-48) (ben@decadent.org.uk) (gcc version 4.3.5 (Debian 4.3.5-4) ) #1 Fri Feb 15 21:38:11 UTC 2013
18:02:49 DEBUG| [    0.000000]
18:02:49 DEBUG| [    0.000000] LINUX started...
18:02:49 DEBUG| [    0.000000] bootconsole [early0] enabled
18:02:49 DEBUG| [    0.000000] CPU revision is: 000182a0 (MIPS 20Kc)
18:02:49 DEBUG| [    0.000000] FPU revision is: 000f8200
18:02:49 DEBUG| [    0.000000] Checking for the multiply/shift bug... no.
18:02:49 DEBUG| [    0.000000] Checking for the daddiu bug... no.
18:02:49 DEBUG| [    0.000000] Determined physical RAM map:
18:02:49 DEBUG| [    0.000000]  memory: 0000000000001000 @ 0000000000000000 (reserved)
18:02:49 DEBUG| [    0.000000]  memory: 00000000000ef000 @ 0000000000001000 (ROM data)
18:02:49 DEBUG| [    0.000000]  memory: 0000000000659000 @ 00000000000f0000 (reserved)
18:02:49 DEBUG| [    0.000000]  memory: 00000000078b7000 @ 0000000000749000 (usable)
18:02:49 DEBUG| [    0.000000] Wasting 104440 bytes for tracking 1865 unused pages
18:02:49 DEBUG| [    0.000000] Initrd not found or empty - disabling initrd
18:02:49 DEBUG| [    0.000000] Zone PFN ranges:
18:02:49 DEBUG| [    0.000000]   DMA      0x00000000 -> 0x00001000
18:02:49 DEBUG| [    0.000000]   Normal   0x00001000 -> 0x00008000
18:02:49 DEBUG| [    0.000000] Movable zone start PFN for each node
18:02:49 DEBUG| [    0.000000] early_node_map[1] active PFN ranges
18:02:49 DEBUG| [    0.000000]     0: 0x00000000 -> 0x00008000
18:02:49 DEBUG| [    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32320
18:02:49 DEBUG| [    0.000000] Kernel command line: printk.time=1 panic=-1 console=ttyS0
18:02:49 DEBUG| Shutting down VM appliance; timeout=30
18:02:49 DEBUG| Attempting graceful termination
18:02:49 DEBUG| Closing console file
18:02:49 DEBUG| Closing console socket
18:02:49 DEBUG| Politely asking QEMU to terminate
...

18:02:49 DEBUG| Transitioning from 'Runstate.CONNECTING' to 'Runstate.RUNNING'.
18:02:49 DEBUG| Opening console file
18:02:49 DEBUG| Opening console socket
18:02:49 DEBUG| [    0.000000] Initializing cgroup subsys cpuset
18:02:49 DEBUG| [    0.000000] Initializing cgroup subsys cpu
18:02:49 DEBUG| [    0.000000] Linux version 2.6.32-5-5kc-malta (Debian 2.6.32-48) (ben@decadent.org.uk) (gcc version 4.3.5 (Debian 4.3.5-4) ) #1 Fri Feb 15 21:38:11 UTC 2013
18:02:49 DEBUG| [    0.000000]
18:02:49 DEBUG| [    0.000000] LINUX started...
18:02:49 DEBUG| [    0.000000] bootconsole [early0] enabled
18:02:49 DEBUG| [    0.000000] CPU revision is: 000182a0 (MIPS 20Kc)
18:02:49 DEBUG| [    0.000000] FPU revision is: 000f8200
18:02:49 DEBUG| [    0.000000] Checking for the multiply/shift bug... no.
18:02:49 DEBUG| [    0.000000] Checking for the daddiu bug... no.
18:02:49 DEBUG| [    0.000000] Determined physical RAM map:
18:02:49 DEBUG| [    0.000000]  memory: 0000000000001000 @ 0000000000000000 (reserved)
18:02:49 DEBUG| [    0.000000]  memory: 00000000000ef000 @ 0000000000001000 (ROM data)
18:02:49 DEBUG| [    0.000000]  memory: 0000000000659000 @ 00000000000f0000 (reserved)
18:02:49 DEBUG| [    0.000000]  m
18:04:48 ERROR| 
18:04:48 ERROR| Reproduced traceback from: /builds/stsquad/qemu/build/pyvenv/lib/python3.10/site-packages/avocado/core/test.py:770
18:04:48 ERROR| Traceback (most recent call last):
18:04:48 ERROR|   File "/builds/stsquad/qemu/build/tests/avocado/replay_kernel.py", line 147, in test_mips64el_malta
18:04:48 ERROR|     self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5)
18:04:48 ERROR|   File "/builds/stsquad/qemu/build/tests/avocado/replay_kernel.py", line 78, in run_rr
18:04:48 ERROR|     t2 = self.run_vm(kernel_path, kernel_command_line, console_pattern,
18:04:48 ERROR|   File "/builds/stsquad/qemu/build/tests/avocado/replay_kernel.py", line 61, in run_vm
18:04:48 ERROR|     self.wait_for_console_pattern(console_pattern, vm)
18:04:48 ERROR|   File "/builds/stsquad/qemu/build/tests/avocado/boot_linux_console.py", line 52, in wait_for_console_pattern
18:04:48 ERROR|     wait_for_console_pattern(self, success_message,
18:04:48 ERROR|   File "/builds/stsquad/qemu/build/tests/avocado/avocado_qemu/__init__.py", line 199, in wait_for_console_pattern
18:04:48 ERROR|     _console_interaction(test, success_message, failure_message, None, vm=vm)
18:04:48 ERROR|   File "/builds/stsquad/qemu/build/tests/avocado/avocado_qemu/__init__.py", line 148, in _console_interaction
18:04:48 ERROR|     msg = console.readline().decode().strip()
Edited by Thomas Huth
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information