Compute refresh events behind the compute flags
What does this MR do and why?
The project authorizations refresh queue has only logged and dropped project and user events so far. This lets it recalculate them instead, so it can start taking over from the Sidekiq refreshes, which can lose updates.
Two flags control which events are recalculated, so each can roll out gradually and be turned off quickly:
authz_refresh_compute_projects: project changes, checked per project.authz_refresh_compute_memberships: membership and user changes, checked per user. It only applies once the projects flag is on for every project, so the two roll out one after the other.
Neither does anything until authz_refresh_write_path is fully on. That flag moves the old Sidekiq refreshes onto the same row locks, so the old and new writers can't overwrite each other with stale results. Its per-project part is still in !269367 (merged), so these flags must not be enabled before that merges and is fully rolled out.
A recalculation that runs out of time puts its events back with the position it reached, and the next run carries on from there. The old Sidekiq refreshes keep running alongside this until their producers are switched off in later work.
References
- Closes https://gitlab.com/gitlab-org/gitlab/-/work_items/637587
- Parent: https://gitlab.com/gitlab-org/gitlab/-/work_items/637561
- Epic: https://gitlab.com/groups/gitlab-org/-/work_items/24326
- Shared write path for per-project refreshes: !269367 (merged)
- Proof of concept: !266827
Screenshots or screen recordings
Not applicable, no UI change.
How to set up and validate locally
- In the Rails console, enable the flags and queue an event for a project:
Feature.enable(:authz_refresh_write_path) Feature.enable(:authz_refresh_compute_projects) project = Project.first ProjectAuthorization.where(project: project).delete_all Authz::ProjectAuthorizations::RefreshEvent.enqueue(:project, [project.id]) Authz::ProjectAuthorizations::RefreshService.new(Gitlab::Metrics::RuntimeLimiter.new(5.minutes)).execute.payload - The payload shows the event under
finished_counts, the project's authorizations are back, andlog/application_json.loghas aRecalculated project authorization refresh eventsline with the rows written.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.