Support data isolation on sec databases
Data isolation is now based on injecting an organization_id sub query, something like:
SELECT * FROM "issues"
WHERE "issues"."namespace_id" IN (
SELECT "namespaces"."id" FROM "namespaces" WHERE "namespaces"."organization_id" = 1000
)These queries don't work on sec if the sharding key is project_id or namespace_id: there is no mirror of namespaces and projects
Boring solution:
Extend the existing syncing CI solution so it will work with sec database.
- Create
sec_project_mirrorsandsec_namespace_mirrorstables - Update these tables in the same process as
ci_project_mirrorsandci_namespace_mirrors - Backfill existing project and namespace data
- Duplicate or refactor
CiNamespaceMirrorsConsistencyCheckWorkerandDatabase::CiProjectMirrorsConsistencyCheckWorkerso it will work with sec database as well - After backfill is done: refactor Data Isolation work so it supports this table
Data retention
Since this data is related to projects and namespaces, the data retention policy is 'infinite'
Edited by Rutger Wessels