Skip to content

Add search spec benchmarking and turn off curation in specs

What does this MR do and why?

I've noticed locally that search related specs are taking an increasingly longer time to run (some over a minute). In the elasticsearch.log, I saw that it takes about 2 seconds for each index curation to complete (creating new indices is expensive). I propose turning this off in specs for now as the curation work is currently on hold and it's not enabled in production.

This MR:

  • adds a new environment variable that can be used to benchmark spec setup times locally
  • adds a name_suffix to indices created. this is to avoid issues with name collisions. currently index names use Date, Hour, and Minute and occasionally there are test failures for specs that run too close together
  • set number of replicas for all standalone indices. We do this already for the main index, and should help spec performance
  • turns off curation for all :elastic* traits
  • adds a new elastic_curation trait to allow curation to be used if needed (one spec was too hard to change)

Screenshots or screen recordings

Before

➜ SEARCH_SPEC_BENCHMARK=1 bundle exec rspec ee/spec/lib/elastic/latest/merge_request_class_proxy_spec.rb
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 3.629528 seconds

Elastic::Latest::MergeRequestClassProxy
  # order random
{"name":"delete_indices!","elapsed_time":0.03336600001784973}
{"name":"create_migrations_index","elapsed_time":0.4440969999996014}
{"name":"mark_all_as_completed!","elapsed_time":3.0783320000045933}
{"name":"create_empty_index","elapsed_time":1.3160849999985658}
{"name":"create_standalone_indices","elapsed_time":14.394169999985024}
{"name":"curate","elapsed_time":19.68056599999545}
  #elastic_search
    search on basis of hidden attribute
      when feature_flag hide_merge_requests_from_banned_users is disabled
        includes merge_requests from the banned authors
      when feature_flag hide_merge_requests_from_banned_users is enabled
        when current_user is non admin
          does not include merge_requests from the banned authors
          has the correct named queries
          when include_archived is set
            does not have a filter for archived
        when current_user is anonymous
          does not include merge_requests from the banned authors
        when current_user is admin
          includes merge_requests from the banned authors

Finished in 51.34 seconds (files took 14.72 seconds to load)
6 examples, 0 failures

Randomized with seed 57090

After

➜ SEARCH_SPEC_BENCHMARK=1 bundle exec rspec ee/spec/lib/elastic/latest/merge_request_class_proxy_spec.rb
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 3.25435 seconds

Elastic::Latest::MergeRequestClassProxy
  # order random
{"name":"delete_indices!","elapsed_time":0.04959500001859851}
{"name":"create_migrations_index","elapsed_time":0.39029300000402145}
{"name":"mark_all_as_completed!","elapsed_time":2.8657789999851957}
{"name":"create_empty_index","elapsed_time":1.2969490000104997}
{"name":"create_standalone_indices","elapsed_time":13.207468999986304}
  #elastic_search
    search on basis of hidden attribute
      when feature_flag hide_merge_requests_from_banned_users is disabled
        includes merge_requests from the banned authors
      when feature_flag hide_merge_requests_from_banned_users is enabled
        when current_user is non admin
          does not include merge_requests from the banned authors
          has the correct named queries
          when include_archived is set
            does not have a filter for archived
        when current_user is anonymous
          does not include merge_requests from the banned authors
        when current_user is admin
          includes merge_requests from the banned authors

Finished in 27.77 seconds (files took 14.56 seconds to load)
6 examples, 0 failures

Randomized with seed 60656

How to set up and validate locally

This only affects specs, you can test the benchmark tool by running any spec that uses one of the elastic traits (:elastic, :elastic_clean, :elastic_delete_by_query or :elastic_with_curation)

SEARCH_SPEC_BENCHMARK=1 bundle exec rspec <PATH_TO_SPEC>

If you want to test the curation changes, run the spec in master then in this branch to see the time differences

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Terri Chu

Merge request reports