Move the IP address for GitLab.com and GitLab Pages
What are we going to do?
We're moving the public IP addresses of GitLab.com and GitLab Pages from the old HAProd
virtual network to GitLabProd
.
Why are we doing it?
Currently the vast majority of GitLab.com runs in the GitLabProd
virtual network. Since the load balancers are in a different one we are liable of paying for peering traffic between the two networks. This change will enable the new load balancers in the new network to start receiving traffic so we can stop paying for peering. In addition to that, the new load balancers are entirely managed by Terraform, which makes them much more manageable.
When are we going to do it?
- Start time: 18 June at 16:00 UTC
- Duration: 30 minutes
- Estimated end time: 18 June at 16:30 UTC
How are we going to do it?
We are moving the public IP address using the Azure API.
How are we preparing for it?
We have successfully moved a public IP address from one virtual network to another. The process takes about 10 to 15 minutes.
What can we check before starting?
We created more load balancers to sustain the traffic. We tested the move and it worked.
What can we check afterwards to ensure that it's working?
GitLab.com must be online.
Impact
- Type of impact: User facing.
- What will happen: GitLab.com will be unreachable.
- Do we expect downtime? (set the override in pagerduty): Yes.
How are we communicating this to our customers?
Since this is part of a more important change to the db the communication is taking place there.
What is the rollback plan?
We move back the IP addresses to the old virtual network.
Monitoring
- Graphs to check for failures:
- Graphs to check for improvements:
- No performance improvement is expected.
- Alerts that may trigger:
- Pretty much everything we've got.
Google Doc to follow during the change
Timeline
- 16:04 - The public IP address of GitLab Pages begins its journey to the new network.
- 16:14 - Pages public IP address attached to the new load balancer.
- 16:23:30 - The public IP address of GitLab.com begins its journey to the new network.
- 16:36:00 - GitLab.com public IP address attached to the new load balancer.
- 16:50:40 - The gitlab-pages daemon is started following the DB maintenance completion. GitLab Pages is back up.