Skip to content

Use elasticsearch for filtered searches of issues and merge requests if available

Problem to solve

Elasticsearch integration provides improved search performance and features when people use the global, group, and project search endpoints. The index is large and expensive to maintain, so it's important we get the most value out of it that we can.

Searches performed via the dashboard, or via the group and project issues, merge_requests and epics interfaces, are always backed by database, rather than elasticsearch, operations. This makes them slower than they could be when the data is in ES, and means some features - notably "find all issues with the term X in the comments for that issue" - are simply too slow to be implemented: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24458#note_180008093

Intended users

Developers, RMs, bug reporters, basically everyone who searches for issues in GitLab

Proposal

Allow the "filtered search" endpoints to use elasticsearch as a datasource in addition to the database. If ES integration is enabled, they would make use of it to satisfy incoming search requests.

This implies improving the Gitlab::Elastic::{,Group,Project}SearchResults classes such that they can construct more-complex elasticsearch queries to apply requirements like "only issues with these milestones", or "only comments with this issue ID".

Permissions and Security

All existing permissions will need to be respected with the new backend.

Documentation

Nothing to do in the first iteration

Testing

We need to ensure that all filtered search terms work against either backend

What is the type of buyer?

Elasticsearch integration is currently GitLab Starter

Links / references

Edited by Nick Thomas