Integrate state preservation into propagation worker
What does this MR do and why?
When a group's deletion is propagated to its subgroups and projects, remember what state each of them was in before, so it can be put back if the deletion is cancelled.
Problem: When a group is scheduled for deletion, a background job (Namespaces::StatePropagationWorker) marks all its subgroups and projects as scheduled for deletion too. If the deletion is later cancelled, we need to know what each of them was before. For example, a subgroup that was archived should go back to archived, not become active. Today nothing records that for subgroups and projects updated by the background job.
Solution:
- Before the job marks a subgroup or project as scheduled for deletion, it saves the previous state (active or archived) in
namespace_details.state_metadata(preserved_states). This uses the same format the state machine already uses for the group the user acts on (ADR 006). - When a group is unarchived, those saved states are cleared, since everything goes back to active.
Putting the saved state back on cancel is done in a separate MR, !261536 (merged). The two can merge in either order.
Nothing triggers this background job in production yet (#599305 (closed)), so there is no user-facing change.
Changes
Namespace.overwrite_state_in_batchcan save (preserve_event:) or clear (clear_preserved:) previous states in the same transaction as the state update- Lowers
STATE_PROPAGATION_ITERATOR_BATCH_SIZEfrom 500 to 200, since each update is slow on cold pages Namespaces::StatePropagationServicesaves states when propagating a deletion and clears them on unarchive- Creates a missing
namespace_detailsrow if needed, so no state is lost
References
Closes #599299 (closed) Parent epic: &21922 Architecture decisions: ADR 003 - State Propagation Model, ADR 006 - State Preservation Follow-up: !261536 (merged)
Replaces the closed !243865 (closed) now that the dependencies !241091 (merged) and !242220 (merged) are merged.
Screenshots or screen recordings
Not applicable - backend change with no UI.
Query plans
Schedule deletion (preserve_event: 'schedule_deletion')
- Locks the overwritable namespaces first, in the same order as a regular transition (namespaces, then namespace_details), to avoid deadlocks. Also reads each row's old state.
SELECT "namespaces"."id", "namespaces"."state" FROM "namespaces" WHERE "namespaces"."state" IN (0, 1) AND "namespaces"."id" IN (<ids>) ORDER BY "namespaces"."id" ASC LIMIT 200 FOR NO KEY UPDATE;Plan: https://console.postgres.ai/gitlab/projects/gitlab-production-main/sessions/64840/commands/169810
- Creates any missing namespace_details rows for the locked namespaces.
INSERT INTO "namespace_details" ("namespace_id","created_at","updated_at") VALUES (<locked id>, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP), ... ON CONFLICT ("namespace_id") DO NOTHING RETURNING "namespace_id";Plan: https://console.postgres.ai/gitlab/projects/gitlab-production-main/sessions/64840/commands/169811
- Saves the previous state. This runs once per old state found (shown for archived).
UPDATE "namespace_details" SET state_metadata = jsonb_set(state_metadata, '{preserved_states}', COALESCE(state_metadata -> 'preserved_states', '{}'::jsonb) || jsonb_build_object('schedule_deletion', 'archived'::text), true) WHERE "namespace_details"."namespace_id" IN (<locked ids>);Plan: https://console.postgres.ai/gitlab/projects/gitlab-production-main/sessions/64840/commands/169812
- Moves the locked namespaces to deletion_scheduled.
UPDATE "namespaces" SET "state" = 2 WHERE "namespaces"."id" IN (<locked ids>);Plan: https://console.postgres.ai/gitlab/gitlab-production-main/sessions/64840/commands/169813
Unarchive (clear_preserved: true)
- Takes the same lock, for archived rows.
SELECT "namespaces"."id", "namespaces"."state" FROM "namespaces" WHERE "namespaces"."state" = 1 AND "namespaces"."id" IN (<ids>) ORDER BY "namespaces"."id" ASC LIMIT 200 FOR NO KEY UPDATE;Plan: https://console.postgres.ai/gitlab/gitlab-production-main/sessions/64840/commands/169814
- Clears the saved states.
UPDATE "namespace_details" SET state_metadata = state_metadata - 'preserved_states' WHERE "namespace_details"."namespace_id" IN (<locked ids>) AND ((state_metadata -> 'preserved_states') IS NOT NULL);Plan: https://console.postgres.ai/gitlab/gitlab-production-main/sessions/64840/commands/169815
- Moves them to ancestor_inherited.
UPDATE "namespaces" SET "state" = 0 WHERE "namespaces"."id" IN (<locked ids>);Plan: https://console.postgres.ai/gitlab/projects/gitlab-production-main/sessions/64840/commands/169818
How to set up and validate locally
-
Set up a group with an archived subgroup, an active subgroup, and a pending propagation record:
group = Group.create!(name: 'Preserve test', path: "preserve-test-#{SecureRandom.hex(4)}", organization: Organizations::Organization.first) archived_subgroup = Group.create!(name: 'Archived sub', path: "archived-sub-#{SecureRandom.hex(4)}", parent: group, organization: group.organization) active_subgroup = Group.create!(name: 'Active sub', path: "active-sub-#{SecureRandom.hex(4)}", parent: group, organization: group.organization) archived_subgroup.update_column(:state, Namespace.states[:archived]) group.update_column(:state, Namespace.states[:deletion_scheduled]) Namespaces::StatePropagation.create!(namespace: group, source_state: :ancestor_inherited, target_state: :deletion_scheduled) -
Run the worker:
Namespaces::StatePropagationWorker.new.perform(group.id, 'deletion_scheduled') -
Check the saved states:
archived_subgroup.reload.state # => "deletion_scheduled" archived_subgroup.state_metadata['preserved_states'] # => {"schedule_deletion"=>"archived"} active_subgroup.reload.state_metadata['preserved_states'] # => {"schedule_deletion"=>"ancestor_inherited"}
You can also run the specs:
bundle exec rspec spec/models/concerns/namespaces/stateful_spec.rb spec/workers/namespaces/state_propagation_worker_spec.rbMR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.