Move gitlab-foss and gitlab to file-cny-01
Production Change - Criticality 2 C2
| Change Objective | Describe the objective of the change |
|---|---|
| Change Type | ConfigurationChange |
| Services Impacted | gitlab-org/gitlab and gitlab-org/gitlab-foss (gitaly) |
| Change Team Members | @jarv |
| Change Severity | C2 |
| Change Reviewer | TBD |
| Tested in staging | This will be run in canary |
| Dry-run output | If the change is done through a script, it is mandatory to have a dry-run capability in the script, run the change in dry-run mode and output the result |
| Due Date | Date and time in UTC timezone for the execution of the change, if possible add the local timezone of the engineer executing the change |
| Time tracking | To estimate and record times associated with changes ( including a possible rollback ) |
Overview
To alleviate recent performance issues on file-02 https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/8407 we will be moving gitlab-org/gitlab-foss and gitlab-org/gitlab to the new file-cny-01 file server. This is a dedicated file node for gitaly, that will eventually be added to the canary fleet. At this time, it will be treated like any other file node.
Monitoring
- Gitaly overview https://dashboards.gitlab.net/d/gitaly-main/gitaly-overview?orgId=1&from=now-1h&to=now
- Gitaly overview for the canary stage https://dashboards.gitlab.net/d/gitaly-main/gitaly-overview?orgId=1&from=now-1h&to=now&var-PROMETHEUS_DS=Global&var-environment=gprd&var-stage=cny&var-sigma=2
- Web overview https://dashboards.gitlab.net/d/web-main/web-overview?orgId=1&%3ForgId=1&from=now-1h&to=now
- General SLA https://dashboards.gitlab.net/d/general-slas/general-slas?orgId=1&from=now-1h&to=now
Procedure
Preparation
Add the new storage shard in the configuration:
-
Add the new node in terraform https://ops.gitlab.net/gitlab-com/gitlab-com-infrastructure/merge_requests/1215 -
Add a new canary role for the file server https://ops.gitlab.net/gitlab-cookbooks/chef-repo/merge_requests/2280 -
Add the file server to the base configuration so it is in the shard list https://ops.gitlab.net/gitlab-cookbooks/chef-repo/merge_requests/2282
GITLAB-FOSS
gitlab-foss is on file-08, we are moving this project first as it should give us a general idea for the time to migrate gitlab-org/gitlab
-
Announce that gitlab-fosswill be switched to read-only for project migration in the following channels:
#announcements
:exclamation: IMPORTANT ANNOUNCEMENT :exclamation:
To alleviate performance issues on one of the gitlab.com storage servers we are going to migrate the
repository for the https://gitlab.com/gitlab-org/gitlab project to a new storage server. During the migration,
the project repository will be set to read-only, meaning that we will not be able to do any write activities
on the repo.
We will start with a migration of `gitlab-org/gitlab-foss`, this migration is starting now and we do not expect it to have any impact on activies.
See https://gitlab.com/gitlab-com/gl-infra/production/issues/1440 for more information
-
Switch the gitlab-foss project to read-only from the rails console
p = Project.find_by_full_path('gitlab-org/gitlab-foss')
p.repository_read_only
-
Initiate the move of gitlab-foss
curl --request PUT --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" -d repository_storage=nfs-file-cny01 https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-foss
-
Monitor sidekiq for updates ProjectUpdateRepositoryStorageWorkerhttps://log.gitlab.net/goto/7ba4c76a63059a399228187bfe6ce88f -
Ensure the migration is complete by validating the shard configuration
p = Project.find_by_full_path('gitlab-org/gitlab-foss')
p.repository_storage
GITLAB
-
Announce that gitlabwill be switched to read-only for project migration in the following channels:
#announcments#backend#company-announcements
:exclamation: IMPORTANT ANNOUNCEMENT :exclamation:
To alleviate performance issues on one of the gitlab.com storage servers we are going to migrate the
repository for the https://gitlab.com/gitlab-org/gitlab project to a different server. During the migration,
the project repository will be set to read-only, meaning that we will not be able to do any write activities
on the repo.
This migration is starting now and we will expect it to take XX minutes.
See https://gitlab.com/gitlab-com/gl-infra/production/issues/1440 for more information
-
Switch the gitlab project to read-only from the rails console
p = Project.find_by_full_path('gitlab-org/gitlab')
p.repository_storage
p.repository_read_only
-
Initiate the move of gitlab
curl --request PUT --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" -d repository_storage=nfs-file-cny01 https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab
-
Monitor sidekiq for updates ProjectUpdateRepositoryStorageWorkerhttps://log.gitlab.net/goto/7ba4c76a63059a399228187bfe6ce88f -
Ensure the migration is complete by validating the shard configuration
p = Project.find_by_full_path('gitlab-org/gitlab')
p.repository_storage
Rollback
-
Initiate a migration back to file-08 for gitlab-foss
curl --request PUT --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" -d repository_storage=nfs-file08 https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-foss
-
Initiate a migration back to file-02 for gitlab
curl --request PUT --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" -d repository_storage=nfs-file02 https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab
Edited by John Jarvis