Delete invalid epic_issue records
What does this MR do?
In https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/1626 we fixed the problem that some issues were assigned to an epic while the issues projects did not belong under the epic group (or its descendants). Now the issues are not displayed anymore so the security problem was fixed.
In this MR we delete those invalid records.
Follow-up
In the next MR validation on EpicIssue
model will be added.
Database
Schedule migration
== 20211023102243 ScheduleDeleteInvalidEpicIssues: migrating ==================
-- Scheduled 1 DeleteInvalidEpicIssues jobs with a maximum of 10000 records per batch and an interval of 120 seconds.
The migration is expected to take at least 120 seconds. Expect all jobs to have completed after 2021-10-26 20:43:42 UTC."
== 20211023102243 ScheduleDeleteInvalidEpicIssues: migrated (3.6027s) =========
Queries with the execution plans
SELECT "epics"."id" FROM "epics" ORDER BY "epics"."id" ASC LIMIT 1
https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/7428/commands/26350
SELECT "epics"."id" FROM "epics" WHERE "epics"."id" >= 1 ORDER BY "epics"."id" ASC LIMIT 1 OFFSET 10000
https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/7428/commands/26351
SELECT MIN("epics"."id"), MAX("epics"."id") FROM "epics" WHERE "epics"."id" >= 1 AND "epics"."id" < 10001
https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/7428/commands/26352
Followed by the next batch with: SELECT "epics"."id" FROM "epics" WHERE "epics"."id" >= 10001 ORDER BY "epics"."id" ASC LIMIT 1 OFFSET 10000
Migration
Queries
Getting epics
SELECT "epics".* FROM "epics" WHERE "epics"."id" BETWEEN 1 AND 1001 ORDER BY "epics"."id" ASC LIMIT 1000
Query plan: https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/7428/commands/26354
Getting epic associations
SELECT "epic_issues".* FROM "epic_issues" WHERE "epic_issues"."epic_id" IN ()
SELECT "issues".* FROM "issues" WHERE "issues"."id" IN ()
SELECT "projects".* FROM "projects" WHERE "projects"."id" IN ()
Getting group & ancestors for every epic
WITH RECURSIVE "base_and_ancestors" AS ((SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."id" = 9970)
UNION
(SELECT "namespaces".* FROM "namespaces", "base_and_ancestors" WHERE "namespaces"."id" = "base_and_ancestors"."parent_id")) SELECT "namespaces".* FROM "base_and_ancestors" AS "namespaces"
Query plan: https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/7056/commands/24921
Deleting epic_issue records
DELETE FROM "epic_issues" WHERE "epic_issues"."id" IN (18, 20, 22)
Counts
- Epics with epic issues: around 73 500
- All epics: around 147 000
- All epic issues: almost 750 000
MR acceptance checklist
-
I have evaluated the MR acceptance checklist for this MR.
Related to #339514