Semantic Code Search: shorten incremental indexing delay to 1 min

What does this MR do and why?

In Semantic Code Search: debounce push-triggered (... (!265341 - merged), we introduced a debounce to push-triggered indexing, delaying incremental indexing by 2 minutes in order to "fold" pushes that are done within that time frame. Initial investigations showed that the 2-minute delay would reduce the number of indexing runs by ~58% on the busiest repositories. It was rolled out globally in production, with actual results showing that the number of indexing runs was reduced by 20% across all repositories, and the additional benefit of reducing the share of jobs deferred due to database health check (see https://gitlab.com/gitlab-org/gitlab/-/work_items/635319#note_4207531781).

We estimated that a delay of 1 minute would reduce the indexing runs by 43% on the busiest repositories. This is a significant decrease with half the delay time, so this is an alternative we should try out. We also need to compare the reduction across all repositories.

This change will be rolled out behind a Feature Flag (active_context_code_push_indexing_delay_1m), after which we will compare metrics for no delay vs 1 minute delay vs 2 minute delay.

References

Screenshots or screen recordings

N/A

How to set up and validate locally

N/A - tests should cover validation

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #635319

Edited by Pam Artiaga

Merge request reports

Loading
Loading