Skip to content

[gstg] Enable the usage new redis-sessions instance

Note: the actual switch to using the instances is behind the FF.
I created this issue similar to #5631 (closed), except that it doesn't include the actual usage of the redis-sessions instance, as it will be the part of the FF rollout.

FF rollout issue: scalability#1429

GPRD change issue: #5969 (closed)

Production Change

Change Summary

This change will configure the Redis-Sessions instance.
scalability#1309

This change is for staging and exists to write & test the process for the equivalent production change.

Change Details

  1. Services Impacted - ServiceRedis
  2. Change Technician - @igorwwwwwwwwwwwwwwwwwwww
  3. Change Reviewer - @nmilojevic1 / @alipniagov
  4. Time tracking - 35min
  5. Downtime Component - None

Detailed steps for the change

Pre-Change Steps - steps to be completed before execution of the change

Estimated Time to Complete (mins) - 1 min

Change Steps - steps to take to execute the change

Estimated Time to Complete (mins) - 30 minutes

  • Merge https://gitlab.com/gitlab-com/gl-infra/chef-repo/-/merge_requests/983
  • From a local copy of chef-repo, run ./bin/gkms-vault-edit gitlab-omnibus-secrets gstg and add an entry for redis_sessions_instance alongside the existing redis configs; use the same password, just adjust the identifier at the end.
  • gitlab-com/gl-infra/k8s-workloads/gitlab-com!1376 (merged)
    • NB: depends on the chef MR being merged first to have the expected effect; do not re-arrange the order.
  • Monitor/wait for the k8s pipeline to complete the staging deploy; it is ok to continue with the rest of the change while the QA-on-staging and production jobs are running.

Post-Change Steps - steps to take to verify the change

Estimated Time to Complete (mins) - 2 minutes

We plan to start using the instance via enabling the FF, so this is not part of this change issue (we'll use FF rollout issue). Here, to validate, we should concentrate on two things:

  • Make sure the main Redis instance is healthy. We don't expect any user-invoked queries into the redis-sessions until the FF is off.
  • Make sure that the redis-sessions is alive and we could connect to that.

Rollback

Rollback steps - steps to be taken in the event of a need to rollback this change

Estimated Time to Complete (mins) - 30 min

  • Disable the feature flag use_multi_store, if we enabled that previously (/chatops run feature set use_multi_store false --staging)
  • Set ENV var GITLAB_USE_REDIS_SESSIONS_STORE to false and restart (true by default)
  • If necessary (the presence of the configuration is the problem), revert the k8s and chef MRs and apply.

Monitoring

Key metrics to observe

Summary of infrastructure changes

  • Does this change introduce new compute instances?
  • Does this change re-size any existing compute instances?
  • Does this change introduce any additional usage of tooling like Elastic Search, CDNs, Cloudflare, etc?

None

Changes checklist

  • This issue has a criticality label (e.g. C1, C2, C3, C4) and a change-type label (e.g. changeunscheduled, changescheduled) based on the Change Management Criticalities.
  • This issue has the change technician as the assignee.
  • Pre-Change, Change, Post-Change, and Rollback steps and have been filled out and reviewed.
  • This Change Issue is linked to the appropriate Issue and/or Epic
  • Necessary approvals have been completed based on the Change Management Workflow.
  • Change has been tested in staging and results noted in a comment on this issue.
  • A dry-run has been conducted and results noted in a comment on this issue.
  • SRE on-call has been informed prior to change being rolled out. (In #production channel, mention @sre-oncall and this issue and await their acknowledgement.)
  • Release managers have been informed (If needed! Cases include DB change) prior to change being rolled out. (In #production channel, mention @release-managers and this issue and await their acknowledgment.)
  • There are currently no active incidents.
Edited by Igor