GitLab

To fully benefit from the advantages of CI/CD, developers need to follow certain principles. Many of these principles have been introduced in the landmark book Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation and are nowadays widely accepted. One of these principles is:

The build process has to be deterministic. Flaky behavior, e.g., tests that sometimes fail, should be avoided at all cost, because this causes maintenance issues. However, addressing this issue through retries after failures might not only hide the underlying problem, but makes issues also harder to debug when they only occur sometimes.

Problem: We analyzed your project and found that the file .gitlab-ci.yml (line 524) violates this principle. The job Debian.cross.i686-linux-gnu (in stage stage1-testing) is set to be retried in case of failures:

Debian.cross.i686-linux-gnu:
    ...
    retry: 1
    ...

Suggested Fix: Remove retry: 1 from the job definition or consider adding the option when to specify failures cases (e.g., runner_system_failure) in which the job can be retried without hiding flaky tests.

Disclaimer: This issue has been automatically reported by CD-Linter, a tool developed at the University of Zurich that detects CI/CD violations in the GitLab CI/CD pipeline configuration. We are currently evaluating the effectiveness of our tool and we are monitoring this issue.

Please up/downvote this issue to indicate whether you agree/disagree with the report.