Skip to content

Resolve "Allow periodic project authorization refresh jobs to use database replica for reads" [RUN ALL RSPEC] [RUN AS-IF-FOSS]

What does this MR do?

For #292443 (closed), and is implementing the plan that was roughly mentioned in !54836 (comment 515219453)

Premise

Currently we enqueue jobs in batched fashion via AuthorizedProjectUpdate::PeriodicRecalculateWorker using a cron job

The purpose of this job is to check if a specific user has any mismatch in their project_authorization records, and if so, fix this discrepancy by adding or removing the necessary records.

Problem

Since we have many users on GitLab.com, the number of these jobs are high, and all these jobs currently read and write project_authorizations data form the primary database, which is increasing the load on the database.

Approach

With this MR, we are trying to reduce the load on the primary database, by off-loading at least the "read" part of this process to the replica database.

With this change, for a specific user:

  • The worker now asks the replica database - "Does the user require a project authorization refresh?"
  • Based on the data currently in the replica db for this user, it returns a true or false.
  • If true, we enqueue a new job that completely runs on the primary database that checks (again) if the user needs a refresh based on the data in the primary database, and if so completes it by adding or removing records from the primary database.

Screenshots (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Manoj M J

Merge request reports