Skip to content

Keep Elasticsearch traversal_ids up to date

Background

In !107577 (merged), we are tracking changes to the user and membership models to keep elasticsearch up to date.

One of the fields in elasticsearch is namespace_ancestry_ids (to be renamed to traversal_ids). This field is built from user membership and namespace ancestry:

def search_membership_ancestry
  members.includes(:source).flat_map do |member|
    member.source&.elastic_namespace_ancestry
  end
end

Proposal

We do track when a membership is added or deleted, but we don't track when the ancestry changes for a membership.

Keep track of changes to traversal_ids and update elasticsearch when needed for all indices that have the field. If there is a change to a namespace's ancestry, we need to update all documents under that namespace too.

Other info:

  • There is an ElasticsearchIndexedNamespace model and Gitlab::Elastic::ElasticsearchEnabledCache which could be useful.

Importance and priority

This data is used to determine which users to show when a group or project is selected. If the data is out of date, there might be unexpected results when searching for users.

The work should be released fairly soon after:

  • The advanced_user_search feature flag has been released to all users.
  • Group and project filtering has been added for advanced search (revert !106972 (merged)).
  • A reindex has been done on the users index
Edited by Madelein van Niekerk