Skip to content

Risk Map for Elasticsearch

Topic to Evaluate

This issue is to describe the facets where Elasticsearch can be evaluated on quality, to document how we are testing for that quality, where we are not explicitly or implicitly testing, and to inform the team as to what areas we can focus more attention on in an ongoing way.

Some questions:

  1. Should we expand this issue to include basic search facets?
  2. Should this map exist as an issue or be made into a page in the handbook?
  3. How do we evaluate the risk level column? Do we even want to keep it? It could be useful in prioritizing test coverage work in the future.
  4. I'm uncertain how many of these facets are our responsibility, perhaps some of them are the responsibility of the Elasticsearch team, if so, we don't want to duplicate their efforts already.
  5. Should we only be linking to open issues and MRs, or do we want to have some historical context by linking closed issues and MRs?
  6. Should the table below be a spreadsheet?

Facets to Evaluate

Facet Are we explicitly testing this? Where? (add link to code) Are we implicitly testing this? Where and how? Issues/MRs related to this facet Risk level Impact Likelihood
Redundancy, high availability, failover, disaster recovery #8010 Moderate Critical Rare
Shard distribution/skewing #10519, grab#138 Moderate Negligible Likely
Indexing, memory use yes Performance testing Moderate Negligible Likely
Indexing, cpu use yes Performance testing #224629 Moderate Negligible Likely
Indexing, speed yes, end to end tests api tests, elasticsearch_reindexing_spec #227724 (closed), #231425 High Marginal Likely
Indexing, accuracy yes, unit test project_spec !29952 (merged), !37834 (merged) Moderate Critical Unlikely
API, parity planned This issue will test both the UI and API
https://gitlab.com/gitlab-org/gitlab/-/issues/212390 		#229056 (closed) #222373 (closed) High Negligible Certain
api:consistency Moderate Negligible Likely
Search accuracy, ranking, recall yes, unit test snippet_search_results_spec !36774 (merged), #228575 (closed) stemming, pluralization Moderate Negligible Likely
Search, speed yes Performance testing #212798 #229627 #33562 (closed) #229981 #225998 (closed) #243667 (closed) High Marginal Certain
Search, efficiency #225692 (closed) #34457 (closed) High Negligible Certain
Regex, code matching #225171 (closed) High Marginal Certain
Regex, efficiency #224629, https://gitlab.com/gitlab-org/gitlab/-/issues/228855 High Marginal Certain
Version compatibility yes (v6.4.2, and v7.7.0) .gitlab-ci.yml Moderate Negligible Likely
Performance at scale yes UI and API Extreme Critical Likely
Security, data leaks https://gitlab.com/gitlab-org/gitlab/-/issues/37959 https://gitlab.com/gitlab-org/gitlab/-/issues/37979 https://gitlab.com/gitlab-org/gitlab/-/issues/238112 High Marginal Certain
Performance bar / Telemetry yes, unit test elasticsearch_spec, global_search_spec, search_counter_spec, search_spec !36899 (merged) !32111 (merged) Moderate Negligible Unlikely
Integrations, AWS credentials yes, unit test client_test gitlab-elasticsearch-indexer!82 (merged) Moderate Marginal Possible
Operation cost, storage High Marginal Likely
Operation cost, traffic #224472 (closed) High Marginal Certain

Performance testing: https://gitlab.com/gitlab-org/quality/performance/-/blob/master/k6/tests/api/api_v4_search_global.js https://gitlab.com/gitlab-org/quality/performance/-/blob/master/k6/tests/api/api_v4_search_groups.js https://gitlab.com/gitlab-org/quality/performance/-/blob/master/k6/tests/api/api_v4_search_projects.js

  • Determine feasibility of the feature
  • Create issue for implementation or update existing implementation issue description with implementation proposal
  • Set weight on implementation issue
  • If weight is greater than 5, break issue into smaller issues
  • Add task
  • Add task

Risks and Implementation Considerations

Team

Edited by Erick Banks