Add RecalculateUserService for project_authorizations
What does this MR do and why?
Adds Authz::ProjectAuthorizations::RecalculateUserService, which recalculates one user's project_authorizations for the refresh queue (https://gitlab.com/groups/gitlab-org/-/work_items/24326).
Today a membership change recalculates all of the user's access. For a user in 108k groups, that's minutes of database time for every change. This service only recalculates the projects the change can affect:
- A group membership change: the group's reach. That's the projects in its subtree, the projects shared with it, and the same for groups it's invited into.
- A project membership change: just that project.
- A repair: every project the user can have access to, including rows no membership explains.
The projects are taken in sorted slices of 1,000. Each slice is compared with the stored rows without a lock. Only the projects that changed go through WriteBatchService, which recalculates them under the user's lock, so most slices never lock. A runtime limiter stops the service between slices and returns a cursor, and the next run resumes after it.
Before reading any stored rows, the service waits once for anyone writing this user's rows. Otherwise a write still in progress, calculated before the change, could make a project look unchanged.
Nothing calls the service yet, so there's no behavior change.
References
- Closes https://gitlab.com/gitlab-org/gitlab/-/work_items/637571
WriteBatchService: !268492 (merged)- Proof of concept: !266827
Database
It uses existing code: Reach, UserProjectGrants#max_access_levels and ProjectAuthorization.stored_levels_for_user. The only new query shape is the stored-rows read with up to 1,000 project IDs, which WriteBatchService caps at 100. Warm on the clone, it takes 1.8 ms plus 6.2 ms of planning, as an index-only scan.
Stored rows for one 1,000-project slice
SELECT "project_authorizations"."project_id", "project_authorizations"."access_level"
FROM "project_authorizations"
WHERE "project_authorizations"."user_id" = <user_id> AND "project_authorizations"."project_id" IN (<1,000 project IDs>)Index Only Scan using index_project_authorizations_on_project_user_access_level on project_authorizations (cost=0.58..2091.89 rows=39 width=8) (actual time=1.750..1.751 rows=0 loops=1)
Index Cond: ((project_id = ANY ('{…}'::integer[])) AND (user_id = N))
Heap Fetches: 0
Buffers: shared hit=2205
Planning Time: 6.182 ms
Execution Time: 1.780 msBefore and after
Warm database time on a production clone, summed over every read each path runs. Each statement ran twice under EXPLAIN (ANALYZE, BUFFERS) and the second run was kept (planning plus execution). "Today" is the read part of Users::RefreshAuthorizedProjectsService (one full calculation of the user's access), or ProjectRecalculatePerUserService for a project membership. Writes aren't included: both paths write the same changed rows.
| Event | Today | New |
|---|---|---|
| Group membership change, typical users (20 samples), p50 / p90 / max | 9 / 19 / 32 ms | 4 / 9 / 307 ms |
| Group membership change, user in 10k groups | 320 s | 4 ms |
| Group membership change, user in 56k groups | 165 s | 3 ms |
| Same user, the group with the largest reach (201k projects) | 165 s | 6.4 s |
| Group membership change, user in 108k groups | 12.6 s | 3 ms |
| Project membership change (20 samples), p50 / p90 / max | 2 / 4 / 5 ms | 2 / 2 / 2 ms |
| Repair, typical users (10 samples), p50 / p90 / max | 10 / 20 / 32 ms | 42 / 204 / 354 ms |
| Repair, user in 10k groups (57k projects) | 320 s | 2.3 s |
| Repair, user in 56k groups (60k projects) | 165 s | 3.6 s |
| Repair, user in 108k groups (203k projects) | 12.6 s | 9.1 s |
- Today, every change for a heavy user is one statement of 12 s to 5 minutes. The new path's statements took at most 136 ms each, and the runtime limiter can stop between slices.
- The new path is slower only for typical repairs, by tens of milliseconds. A repair first collects every project the user could reach, then compares them in slices. Repairs are rare: the verifier, runbooks and
User#refresh_authorized_projects.
How to set up and validate locally
Not applicable, nothing calls the service yet. The spec covers it.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist.