Skip to content

Prevent Rotating External Secrets Without Follow Process

Summary

We have a specific process on how to rotate/update secrets for external-secrets and not everyone knows about it, which results into folks updating the version directly like we've done in gitlab-com/gl-infra/k8s-workloads/gitlab-com!4084 (merged), which end up causing production#19116 (closed) because we had concurrent deployments.

Related Incident(s)

Originating issue(s): production#19116 (closed)

Desired Outcome/Acceptance Criteria

Having a Danger job that checks changes in external secrets, and warn that we should follow the steps

Associated Services

ServiceVault

Corrective Action Issue Checklist

  • Link the incident(s) this corrective action arose from
  • Give context for what problem this corrective action is trying to prevent re-occurring
  • Assign a severity label (this is the highest sev of related incidents, defaults to 'severity::4')
  • Assign a priority (this will default to 'Production Engineering::P4' but should match the severity of the related incident)
  • Assign a service label
  • Assign a team label