Skip to content

Post-deploy migration SetAllowImmediateNamespacesDeletionToFalseOnSaas failing in Cells due to PG::UndefinedColumn

Summary

The post-deploy migration SetAllowImmediateNamespacesDeletionToFalseOnSaas added in !205554 (merged) failed when the auto-deploy process tried to run it on Cells:

Failing job: https://ops.gitlab.net/gitlab-com/gl-infra/cells/tissue/-/jobs/20443529

Stack trace, relevant section:

Caused by:
PG::UndefinedColumn: ERROR:  column "allow_immediate_namespaces_deletion" of relation "application_settings" does not exist
LINE 1: ...fig_name:main*/ UPDATE "application_settings" SET "allow_imm...
															 ^
/srv/gitlab/db/post_migrate/20250922144106_set_allow_immediate_namespaces_deletion_to_false_on_saas.rb:18:in `up'

This PDM has not yet been executed on GitLab.com's gstg or gprd. As this is caused by a missing column, it seems like a bug in the PDM and the bug might cause a failure on GitLab.com as well. We would like to confirm if this is indeed a bug and fix the PDM if so.

The failure was noticed and reported by @tkhandelwal3 on Slack (internal).

Status of GitLab.com

This PDM was added in the commit 4e8ec857

We ran PDMs in gprd about 10 days ago, at ebdb02de according to https://ops.gitlab.net/gitlab-org/release/metadata/-/environments/923?tab=deployment-history As this PDM was added after the last run, we have not yet run this PDM in Production:

$ git merge-base --is-ancestor 4e8ec857ab9733920034c456795d19959ad4239 ebdb02dee7478a75762ab9befab6ef1f4b2932b9
# exit code: 1

Bug

The regular migration in !205554 (diffs) adds a column called namespace_deletion_settings of type jsonb and then adds a check constraint to ensure that this column is a jsonb: 4e8ec857

But the PDM may be referring to the column allow_immediate_namespaces_deletion: 4e8ec857?

jsonb_accessor :namespace_deletion_settings,
      allow_immediate_namespaces_deletion: [:boolean, { default: true }]

Exit criteria

  • Confirm whether this is actually a bug in the PDM with the PDM author or other team members
  • If this is a bug, fix the PDM in the GitLab codebase
  • If this is not a bug, investigate why it failed on Cells
  • If this is not a bug, investigate whether it is likely to fail on gprd
Edited by 🤖 GitLab Bot 🤖