Add indexes for namespace_traversal_ids
What does this MR do and why?
Add basic namespace_traversal_ids for issues table
This adds the two primary indexes we need for the namespace_traversal_ids:
-
An index on the namespace_traversal_ids which will be used for sub group lookups and sorting will mostly happen in memory.
-
A separate on the root namespace since root namespaces contain a larger dataset. So we need a special index and not sort in memory if possible.
For now only focusing on
created_atsince the query optimizer seems to prefer using the index onupdated_atand filter ontraversal_ids.
Changelog: added
References
- Related to the changes in the WorkItemsFinder !206102 (merged)
- More about the traversal_ids introduction: https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/traversal_ids_on_issues/
Screenshots or screen recordings
In addition to the queries above, I have an automated script which runs a few more queries with different filters in comparison of the old and new query times and buffer usage: https://gitlab.com/gitlab-org/plan-stage/product-planning/issue-database-research-benchmarking/-/blob/main/report.md?ref_type=heads
How to set up and validate locally
Running through https://docs.gitlab.com/development/database/adding_database_indexes/#test-database-index-changes-locally
bundle exec rails gitlab:db:execute_async_index_operations:all
\d issues
..
namespace_traversal_ids | bigint[] | | | '{}'::bigint[]
Indexes:
"issues_pkey" PRIMARY KEY, btree (id)
[...]
"idx_issues_root_namespace_created_at" btree ((namespace_traversal_ids[1]), created_at)
"idx_issues_state_id_namespace_traversal_ids" btree (state_id, namespace_traversal_ids)
...
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.