You need to sign in or sign up before continuing.

Import projects into project_build_artifacts_size_refreshes

Production Change

Change Summary

This is part of the work for rolling out gitlab-org/gitlab#356018 (closed) on production, which was introduced in gitlab-org/gitlab!81306 (merged).

Before we can start enabling the feature flag, we need to prefill the project_build_artifacts_size_refreshes table first. We need to run some commands through the rails console in production.

On the rails console, we will invoke gitlab:refresh_project_statistics_build_artifacts_size which will download a CSV that contains the project IDs that we will then import into the new table. We are expecting around 700K projects to be imported.

Change Details

Services Impacted - ServicePostgres
Change Technician - @iamricecake @reprazent
Change Reviewer - @reprazent
Time tracking - 15m
Downtime Component - N/A

Detailed steps for the change

Pre-Change Steps - steps to be completed before execution of the change

Estimated Time to Complete (mins) - Estimated Time to Complete in Minutes

Set label changein-progress on this issue

Change Steps - steps to take to execute the change

Estimated Time to Complete (mins) - Estimated Time to Complete in Minutes

On the rails console, run:

Rails.application.load_tasks
csv_url='https://storage.googleapis.com/verify_pipeline_insights/recalculate_project_artifact_size.csv'
Rake::Task['gitlab:refresh_project_statistics_build_artifacts_size'].invoke(csv_url)

Take note of the output of the rake task, showing the number of imported projects.

Post-Change Steps - steps to take to verify the change

Estimated Time to Complete (mins) - Estimated Time to Complete in Minutes

Confirm records are created in the new project_build_artifacts_size_refreshes table.
- In the Rails console, we can run:
```
Projects::BuildArtifactsSizeRefresh.remaining.count
```

Rollback

Rollback steps - steps to be taken in the event of a need to rollback this change

Estimated Time to Complete (mins) - Estimated Time to Complete in Minutes

Truncate the project_build_artifacts_size_refreshes table
- In the Rails console, we can run:
```
Projects::BuildArtifactsSizeRefresh.in_batches { |r| r.delete_all; sleep(0.5) }
```

Monitoring

Keep an eye on the size of new table to see progress:

Thanos

Look at the Patroni dashboard: https://dashboards.gitlab.net/d/patroni-main/patroni-overview?orgId=1

Summary of infrastructure changes

Does this change introduce new compute instances?
Does this change re-size any existing compute instances?
Does this change introduce any additional usage of tooling like Elastic Search, CDNs, Cloudflare, etc?

Summary of the above

Change Reviewer checklist

C4 C3 C2 C1:

The scheduled day and time of execution of the change is appropriate.
The change plan is technically accurate.
The change plan includes estimated timing values based on previous testing.
The change plan includes a viable rollback plan.
The specified metrics/monitoring dashboards provide sufficient visibility for the change.

C2 C1:

The complexity of the plan is appropriate for the corresponding risk of the change. (i.e. the plan contains clear details).
The change plan includes success measures for all steps/milestones during the execution.
The change adequately minimizes risk within the environment/service.
The performance implications of executing the change are well-understood and documented.
The specified metrics/monitoring dashboards provide sufficient visibility for the change. - If not, is it possible (or necessary) to make changes to observability platforms for added visibility?
The change has a primary and secondary SRE with knowledge of the details available during the change window.

Change Technician checklist

Edited Apr 04, 2022 by Bob Van Landuyt