Add smaller search curation thresholds for development environment
What does this MR do and why?
This lowers the threshold for search index curation to occur in local development environments. The IndexCurationWorker
will pick this up or curation can be triggered manually via a console.
Screenshots or screen recordings
Here is the output of GET _cat/indices?v
after running ./bin/rake gitlab:elastic:index
then triggering search index curation manually:
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open gitlab-development-users-20221209-0030 sz9aFU8FQL28YCTUzy0RZw 5 1 47 0 65.4kb 65.4kb
yellow open gitlab-development-notes-20221209-0030 BD32bj07RSKd_0gbKi7etA 5 1 937 0 172.3kb 172.3kb
yellow open gitlab-development-notes-20221209-0031 wzNTVUgvQHSRLuiWzUcsEA 5 1 0 0 450b 450b
yellow open gitlab-development-merge_requests-20221209-0031 zn055CQURPOqTgoKXsS54A 5 1 0 0 450b 450b
yellow open gitlab-development-migrations CsSM_OwrTXC7cYf_0Z7Fhg 1 1 26 0 5.9kb 5.9kb
yellow open gitlab-development-merge_requests-20221209-0030 zlImsYhdR6edn5iYmwHYug 5 1 141 0 105.2kb 105.2kb
yellow open gitlab-development-issues-20221209-0031 3QAliytFRviEFPCDlfpxYg 5 1 0 0 450b 450b
yellow open gitlab-development-issues-20221209-0030 TYs-U2VSReC7nilYaunZQA 5 1 461 0 155.6kb 155.6kb
yellow open gitlab-development-commits-20221209-0030 KuiPSXuVSw-ejgcw6kAe8Q 5 1 0 0 1kb 1kb
yellow open gitlab-development-20221209-0030 bzQ0txoYSquqtimT6WgJ5Q 5 1 98 0 116kb 116kb
How to set up and validate locally
Inspect thresholds are lower for development environment
[1] pry(main)> Gitlab::Search::IndexCurator::DEFAULT_SETTINGS
=> {:dry_run=>true,
:debug=>false,
:force=>false,
:max_shard_size_gb=>1,
:max_docs_denominator=>100,
:min_docs_before_rollover=>50,
:max_docs_shard_count=>5,
:ignore_patterns=>[/migrations/],
:include_patterns=>[],
:index_pattern=>"gitlab*"}
Run rollover manually
Gitlab::Search::IndexCurator.curate(dry_run: false)
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by John Mason