Alerting when mirroring fails

Context

Repository mirroring is widely used, most popularly from a canonical repo to a security mirror. Many infrastructure operations function on security mirrors, so we need them to be in sync all the time.

However, the mirroring can fail because of various reasons: diverge ref, expired token, permission, etc. They often fail silently, and we don't know about that until an engineer investigates.

Proposal

  • Send a Slack notification when detecting a failed mirror

Open questions

  • Who is the owner of a mirroring setup? Delivery team, or the stage team
  • Notification medium?