Skip to content

Do a first batch with a mix of different servers before larger batches

In our two attempts to rebuild gprd-cny (gitlab-com/gl-infra/production#6451 (closed) and gitlab-com/gl-infra/production#6456 (closed)) we encountered issues that were not covered in our gstg testing, because of the differences between gstg and gprd. We were able to successfully execute the change on canary production, but it's possible that differences between it an other file nodes may lead to similar hiccups. To mitigate this risk, instead of doing all the nodes of a single type in one or two batch, let's have a first batch that consists of a sample of all our different types of nodes, so that we're able to detect possible failure conditions if they exists while affecting only a single node per type.

Edited by Alejandro Rodríguez

Merge request reports

Loading