Add search spec benchmarking and turn off curation in specs
What does this MR do and why?
I've noticed locally that search related specs are taking an increasingly longer time to run (some over a minute). In the elasticsearch.log
, I saw that it takes about 2 seconds for each index curation to complete (creating new indices is expensive). I propose turning this off in specs for now as the curation work is currently on hold and it's not enabled in production.
This MR:
- adds a new environment variable that can be used to benchmark spec setup times locally
- adds a
name_suffix
to indices created. this is to avoid issues with name collisions. currently index names use Date, Hour, and Minute and occasionally there are test failures for specs that run too close together - set number of replicas for all standalone indices. We do this already for the main index, and should help spec performance
- turns off curation for all
:elastic*
traits - adds a new
elastic_curation
trait to allow curation to be used if needed (one spec was too hard to change)
Screenshots or screen recordings
Before
➜ SEARCH_SPEC_BENCHMARK=1 bundle exec rspec ee/spec/lib/elastic/latest/merge_request_class_proxy_spec.rb
Run options: include {:focus=>true}
All examples were filtered out; ignoring {:focus=>true}
Test environment set up in 3.629528 seconds
Elastic::Latest::MergeRequestClassProxy
# order random
{"name":"delete_indices!","elapsed_time":0.03336600001784973}
{"name":"create_migrations_index","elapsed_time":0.4440969999996014}
{"name":"mark_all_as_completed!","elapsed_time":3.0783320000045933}
{"name":"create_empty_index","elapsed_time":1.3160849999985658}
{"name":"create_standalone_indices","elapsed_time":14.394169999985024}
{"name":"curate","elapsed_time":19.68056599999545}
#elastic_search
search on basis of hidden attribute
when feature_flag hide_merge_requests_from_banned_users is disabled
includes merge_requests from the banned authors
when feature_flag hide_merge_requests_from_banned_users is enabled
when current_user is non admin
does not include merge_requests from the banned authors
has the correct named queries
when include_archived is set
does not have a filter for archived
when current_user is anonymous
does not include merge_requests from the banned authors
when current_user is admin
includes merge_requests from the banned authors
Finished in 51.34 seconds (files took 14.72 seconds to load)
6 examples, 0 failures
Randomized with seed 57090
After
➜ SEARCH_SPEC_BENCHMARK=1 bundle exec rspec ee/spec/lib/elastic/latest/merge_request_class_proxy_spec.rb
Run options: include {:focus=>true}
All examples were filtered out; ignoring {:focus=>true}
Test environment set up in 3.25435 seconds
Elastic::Latest::MergeRequestClassProxy
# order random
{"name":"delete_indices!","elapsed_time":0.04959500001859851}
{"name":"create_migrations_index","elapsed_time":0.39029300000402145}
{"name":"mark_all_as_completed!","elapsed_time":2.8657789999851957}
{"name":"create_empty_index","elapsed_time":1.2969490000104997}
{"name":"create_standalone_indices","elapsed_time":13.207468999986304}
#elastic_search
search on basis of hidden attribute
when feature_flag hide_merge_requests_from_banned_users is disabled
includes merge_requests from the banned authors
when feature_flag hide_merge_requests_from_banned_users is enabled
when current_user is non admin
does not include merge_requests from the banned authors
has the correct named queries
when include_archived is set
does not have a filter for archived
when current_user is anonymous
does not include merge_requests from the banned authors
when current_user is admin
includes merge_requests from the banned authors
Finished in 27.77 seconds (files took 14.56 seconds to load)
6 examples, 0 failures
Randomized with seed 60656
How to set up and validate locally
This only affects specs, you can test the benchmark tool by running any spec that uses one of the elastic traits (:elastic
, :elastic_clean
, :elastic_delete_by_query
or :elastic_with_curation
)
SEARCH_SPEC_BENCHMARK=1 bundle exec rspec <PATH_TO_SPEC>
If you want to test the curation changes, run the spec in master
then in this branch to see the time differences
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.