Publicise that teams should keep an eye on failed pipelines on default branch of security mirrors of auto deploy projects
Summary
For auto deploy projects, release-tools picks a commit for deployment only if it has a successful pipeline. If the latest commit does not have a successful pipeline, and there is no newer commit with a successful pipeline after that, we could be in a situation where the latest commit(s) of the project do not get deployed for many days.
release-tools notifies RMs if an auto deploy branch is behind master
by 7 days, but this is a last-resort measure. Ideally, any failed pipelines should be fixed as soon as possible so that the commit is deployed.
Proposal
Spread the following information to teams that own projects that are auto deployed:
- Failed pipelines on master branch of security mirror can cause commits to not be deployed by auto-deploy
- Master pipelines on the security mirror need to be monitored and fixed. Set up Slack notifications for failed pipelines, or any other monitoring system that the team prefers.