Elasticsearch indexer is slow during initial indexing

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Collaborate/take over this issue

Summary

During working on gitlab-org/search-team/team-tasks#203 (closed) we encountered extremely slow initial indexing of gitlab-org/gitlab (30+ minutes)

Steps to reproduce

What is the current bug behavior?

What is the expected correct behavior?

Relevant logs and/or screenshots

Possible fixes

Apply the same changes introduced in gitlab-zoekt-indexer in gitlab-zoekt-indexer!331 (merged)

Implementation Plan

gitlab-elasticsearch-indexer changes

• Update gitaly to 17.7 (this must include putting gitaly on the no-update list for renovate bot)
• Determine how gitaly will be updated going forward, ideally should stay on the same version as gitlab project (this should probably be a separate issue in the gitlab-elasticsearch-indexer project
• Add new code as an alternative path for gitaly
• Add a new command line argument that will run the new code if true
• Write up an issue to deprecate and remove old code one feature flag is removed from rails

gitlab-org/gitlab changes

• add a new feature flag
• send the feature flag to the indexer as a command line argument
• remove the feature flag