Provision redis-cluster-ratelimiting in gprd
This issue provisions the infrastructure and initializes the Redis nodes into a cluster. It also adds rails configuration to connect and authenticate to the cluster, but it does not enable use of this cluster. Live cutover will happen via feature flag toggle during a change issue.
Ordered list of setup tasks:
-
Create chef roles for the cluster and each shard, and setup the chef-managed prometheus servers to scrape VMs having the new chef role: https://gitlab.com/gitlab-com/gl-infra/chef-repo/-/merge_requests/3067 -
Add GKMS secrets for: -
redis-cluster
: #2256 (comment 1319211093) -
redis-exporter
: #2256 (comment 1319255710)
-
-
Silence alerts for env=gprd AND type=redis-cluster-ratelimiting
. Temporary precaution to avoid false alarms during provisioning: #2256 (comment 1319345511) -
Create VMs via Terraform: https://ops.gitlab.net/gitlab-com/gl-infra/config-mgmt/-/merge_requests/5260 -
Initialize the Redis Cluster, assigning roles and hash slots to the Redis nodes: #2256 (comment 1319368498) -
Add Redis password to Hashicorp Vault for the rails
application user: #2256 (comment 1319379639) -
Configure rails clients to connect to this cluster (but not actively use it until we enable the feature flag): gitlab-com/gl-infra/k8s-workloads/gitlab-com!2619 (merged)
Related:
- Provisioning in
gstg
: #2210 (closed) - Feature flag issue: gitlab-org/gitlab#385681 (closed)
- Production rollout (feature flag toggle): production#8577
Edited by Matt Smiley