CI: retry doctests only on segfaults, increase number of retries to 5

Description

Problem: doctests often segfault due to a bug in GHC, and 3 retries is often not enough.

Solution: increase the number of retries, and change the job to retry only on a segfault, and not on a normal test failure.

Related issue(s)

None

✅ Checklist for your Merge Request

Related changes (conditional)

• Tests (see short guidelines)

  • If I added new functionality, I added tests covering it.
  • If I fixed a bug, I added a regression test to prevent the bug from silently reappearing again.

• Documentation

  • I checked whether I should update the docs and did so if necessary:
    • Root README and other README.md files
    • Haddock
    • docs/
  • I updated changelog files of all affected packages released to Hackage if my changes are externally visible.

Stylistic guide (mandatory)

• My commits comply with the following policy.
• My code complies with the style guide.