CI: add one retry of failed tests
Some tests are flaky, so retrying them normally succeeds. There are pytest
plugins that can retry tests automatically (e.g., flaky, pytest-rerunfailures),
but these don't work with pytest-forked. We need pytest-forked when
running Tango devices.
Instead of using a plugin, the CI scripts now manually try repeating the last failed
tests, using pytest --lf, once. We don't bother with the telemetry specific tests,
since these haven't been flaky.
We now need different names for the junit XML reports, so this is removed from
.pyproject.toml. We manually specify this in the CI scripts.
Click to see example
https://gitlab.com/tango-controls/pytango/-/jobs/7855951936
...
$ pytest || pytest --lf
============================= test session starts ==============================
platform darwin -- Python 3.12.6, pytest-8.3.3, pluggy-1.5.0 -- /Users/gitlab/micromamba/bin/python
...
=================================== FAILURES ===================================
___________________ test_async_attribute_with_callback[poll] ___________________
...
=========================== short test summary info ============================
FAILED tests/test_async.py::test_async_attribute_with_callback[poll]
====== 1 failed, 1265 passed, 31 skipped, 47 xfailed in 108.52s (0:01:48) ======
============================= test session starts ==============================
platform darwin -- Python 3.12.6, pytest-8.3.3, pluggy-1.5.0 -- /Users/gitlab/micromamba/bin/python
cachedir: .pytest_cache
rootdir: /Users/gitlab/builds/tango-controls/pytango
configfile: pyproject.toml
testpaths: tests
plugins: asyncio-0.24.0, cov-5.0.0, forked-1.6.0
asyncio: mode=Mode.STRICT, default_loop_scope=function
collecting ... collected 1 item
run-last-failure: rerun previous 1 failure (skipped 9 files)
tests/test_async.py::test_async_attribute_with_callback[poll] PASSED [100%]
============================== 1 passed in 0.10s ===============================
Edited by Anton Joubert