Pipeline construction of k8s-workloads/gitlab-com needs improvement for environment targetting
Summary
Problem Statement
The pipelines in k8s-workloads/gitlab-com
are not very smart for configuration changes. It's common for chart bumps that target specific environments, where all environments get deployed too regardless of a diff occuring for another environment unrelated to a proposed change. This currently greatly slows down if we need to make a revert to production. This takes a long time because staging and preprod both get deployed too, and subsequently their QA pipelines may run. This situation was the primary reason it took a long time before incident production#6139 (closed) could be remediated.
We have a few lingering proposals to make this repository better overall, but I don't believe we have any that are specific towards making our pipeline better at the moment.
Related Incident(s)
Originating issue(s): production#6139 (closed)
Related Existing Issues
Desired Outcome/Acceptance criteria
When a change target a specific environment, jobs in the CI pipeline are only created that target that environment.
Associated Services
The target service for this reliability~11110745 is any service deployed into Kubernetes as deployed by the repository k8s-workloads/gitlab-com
Corrective Action Issue Checklist
-
link the incident(s) this corrective action arose out of -
give context for what problem this corrective action is trying to prevent from re-occurring -
assign a severity label (this is the highest sev of related incidents, defaults to 'severity::4') -
assign a priority (this will default to 'priority::4')