Skip to content

Enable parallel unit tests on Debian 12 CI and run Debian 12 server tests on 4 vCPU machine

Igor Ponomarev requested to merge enable-parallel-unit-tests-on-debian-12 into master

To use parallel unit tests install python3-pytest-xdist package and pass --numprocesses=auto option to pytest.

This will only work on Debian 12 because pytest-django gained support for pytest-xdist in version 2.5.0

https://pytest-django.readthedocs.io/en/latest/changelog.html#v2-5-0

Debian 11 ships pytest-django 2.2.0

https://packages.debian.org/bullseye/python3-pytest-xdist

Debian 12 ships pytest-django 3.1.0

https://packages.debian.org/bookworm/python3-pytest-xdist

The dispatcher tests instantly became around 30 seconds faster when run in parallel.

Debian 11 dispatcher test (01:18)

Debian 12 parallel dispatcher test (00:52)

However, the server tests did not get that big of the boost. Debian 12 server tests already slower than Debian 11 by about 60%. Using the parallel test the difference narrowed to around 30%.

Debian 11 server test (01:49)

Debian 12 parallel server test (02:26)

I suspect this is because server tests are more CPU bound and the default runner has 2 vCPUs which are probably 2 SMT threads over 1 core.

I switched to a 4 vCPU machine to Debian 12 server tests and now they are faster than Debian 11 tests.

Debian 12 parallel server test on 4 vCPU machine (01:21)

Merge request reports