Skip to content

Set deletion_policy on GSM secret from vault to ABANDON instead of DELETE

Igor requested to merge vault-secret-deletion-policy-abandon into main

We currently re-create secrets and delete the old version. When a deployment creates an unhealthy revision, we keep serving from the old healthy one, but once we try to start new containers on that old revision, they fail to start because the underlying secret has been nuked.

This is a corrective action for gitlab-com/gl-infra/production#18252 (closed).

We saw this problem previously: team#100 (closed), and fixed it: !83 (merged).

But when we introduced vault syncing, we forgot to carry that fix over: !92 (merged).

A few points that aren't quite clear yet:

  • What does ABANDON do exactly? Docs on this seem to be lacking.
  • Do we re-create these secrets on every deploy or only if the secret changes?

Merge request reports