[Feature flag] Rollout of `rebalance_issues`
:rebalance_issues feature flag ...
- Team: Create / Source Code
- Most appropriate slack channel to reach out to:
- Best individual to reach out to: @alexkalderimis
What are we expecting to happen?
Problematic issue sets (ones that show large numbers of timeouts and
NoSpaceLeft exceptions) should begin rebalancing, and the incidence of these errors should decline.
What might happen if this goes wrong?
This may consume a large number of resources in the worker, since this can be a long running worker, which may itself timeout. We should watch for spikes in queue processing time.
What can we monitor to detect problems with this?
We should watch sentry for
NoSpaceLeft and any error in the
IssueRebalancingWorker. We should watch kibana to observe queue sizes:
If applicable, any groups/projects that are happy to have this feature turned on early. Some organizations may wish to test big changes they are interested in with a small subset of users ahead of time for example.
- Beta groups/affected user: ["Project:10152778", "Project:10986423", "Project:11283490", "Project:11638590", "Project:14963009", "Project:18668865", "Project:4326131", "Project:4482940"]
Gitlab has too many issues to use this worker at present - future updates may allow for this.
Roll Out Steps
Enable on staging
Test on staging
Ensure that documentation has been updated
Enable on GitLab.com for individual groups/projects listed above and verify behaviour
Coordinate a time to enable the flag with
Announce on the issue an estimated time this will be enabled on GitLab.com
Enable on GitLab.com by running chatops command in
Cross post chatops slack command to
#support_gitlab-com(more guidance when this is necessary in the dev docs) and in your team channel
Announce on the issue that the flag has been enabled
Remove feature flag and add changelog entry
After the flag removal is deployed, clean up the feature flag by running chatops command in