Auto-deploy can potentially pick a "green" docs-only build in an otherwise broken `master`
In gitlab-org/gitlab
, we've begun running limited pipelines based on the paths changed in a commit. E.g., only running docs-related tests when only changing docs.
However, because this even happens on master
, there's potential for auto-deploy to pick a "green" build that only ran docs linting, and for the rest of the specs (i.e., functionality) to be broken.
Ephemeral chat link: https://gitlab.slack.com/archives/C3JJET4Q6/p1572543341228800
Robert Speicher 4 days ago
I just realized that we're running limited pipelines on master, for example when a commit only has docs changes: https://gitlab.com/gitlab-org/gitlab/pipelines/92669403. This is problematic, because we build auto-deploys based on green masters, and this would've been considered green even though things like RSpec were failing in the commits surrounding this one.
Kamil Trzciński 4 days ago
The new rules: for the rescue on this one.
Kamil Trzciński 4 days ago
Only they allow to model that behavior.
Robert Speicher 14 minutes ago
@remy Can you take a look here please?
Rémy Coutable 5 minutes ago
That is known, that’s why we run full pipelines on master every two hours.
Robert Speicher 4 minutes ago
Ok, but that still creates the danger of the full run being red and auto-deploy picking a "green" docs-only build that's actually still broken :thinking_face:
Rémy Coutable 3 minutes ago
Ideally, you’d only look at scheduled pipelines.
Rémy Coutable 3 minutes ago
@kamil How would rules solve that?
Kamil Trzciński 3 minutes ago
rules:
- branch: master
- changes: [doc/**]
Run on master, always, or create if doc/** is changed.
Kamil Trzciński 3 minutes ago
In todays syntax:
rules:
- if: $CI_COMMIT_REF_NAME=="master"
- changes: [doc/**]
(edited)
Rémy Coutable 2 minutes ago
Yeah, I realized that after asking. :stuck_out_tongue: