Refactor secrets manager deprovisioning to support direct paths in the model and prevent cascading deletion

Dmytro Biryukovrequested to merge
dbiryukov_refactor_secrets_manager_deprovisioning_prevent_cascading_deleting_mr-583654 into master

Related: #583654

What does this MR do and why?

Follow-up to !215116 (merged) improve reliability and resilience of secrets manager deprovisioning, especially in cases where the deprovision worker is abruptly killed and deprovision tasks become stale.

This is the first MR that lays the foundation for a later worker refactoring for the cronbase:

  1. Remove cascading deletion of secrets manager records.
  2. Introduce group_path and project_path fields in the group and project secrets manager tables.
  3. Make these new fields the primary source of truth for OpenBao requests.

A follow-up issue will refactor the deprovision worker to use cronbase.

Problem to solve

Currently, deprovisioning is tied to a worker that can be interrupted (e.g., worker crash, deploy, scaling event). If this happens mid-deprovision, we risk:

  • Stale deprovision tasks that are never retried
  • Inconsistent state between GitLab DB and OpenBao
  • Loss of secrets_manager records due to cascading deletes when projects/groups are removed, making it impossible to correctly clean up external secrets later

This MR prevents database-level cascading deletion of secrets_manager records so that project and group secrets_manager rows persist even after their owning project/group is removed. Keeping these records allows us to reliably track deprovisioning state and retain references to external secrets required for cleanup. We also add new columns to the project and group secrets_manager tables (e.g., namespace_path and the actual OpenBao secret path) to provide stable references that simplify and harden secret cleanup during deprovisioning.

References

Because this is an experiment, we will not backfill existing data for the newly introduced project and group secrets manager fields. The rollout plan is to first add the new columns as nullable, then reprovision once this is in production, and finally enforce non-null constraints on these columns.

Before After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Dmytro Biryukov

Merge request reports