Skip to content

Make slices and scroll configurable in Elasticsearch reindex

Problem to solve

In !34069 (merged) we introduced the ability to reindex your Elasticsearch cluster from within GitLab.

When doing some benchmarking for gitlab-com/gl-infra/production#2408 (comment 383415748) we found that we could substantially increase the reindex speed by tweaking the slices parameter. Additionally in the past we've had problems with the scroll parameter defaulting to too short a time and the reindex fails as the scrolls is deleted mid way through.

In order for us to make use of the reindex option in GitLab we'll need to be able to increase both of these parameters. There won't ever be 1 correct number for all users since there are tradeoffs (eg. higher CPU when higher slices) and the larger number of slices won't be appropriate for small clusters at all.

Intended users

User experience goal

Proposal

We need some way to configure these parameters when triggering the reindex. Perhaps a couple of inputs next to the button or pop up when you click the button before submitting or some other design.

Further details

Permissions and Security

Documentation

Availability & Testing

What does success look like, and how can we measure that?

What is the type of buyer?

Is this a cross-stage feature?

Links / references