Add new fields to projects as_indexed_json (iteration 2)
Problem to solve
The projects search is unable to add filtering options due to missing data fields in the index required.
Before starting the work
Data needs to be gathered to determine how often these fields are updated. We need to evaluate the impact on indexing requests
Note: this work does not block indexing all projects. It's purely to add additional data to the index to allow use in other areas of GitLab (besides search results)
Proposal
We will add new fields in multiple iterations. This will allow us to evaluate the indexing impact with less updates to projects by splitting new fields into two groups.
- Add an Advanced search migration to update the mappings with the new fields
- Add new fields to
ProjectInstanceProxy
, gate by the migration added above- star_count (integer)
- issue_count (integer)
- merge_request_count (integer)
- last_repository_updated_date (date)
- Bump schema_version
- Check if additional preloading is required for indexing (N+1 checks are located in
ee/spec/services/elastic/process_bookkeeping_service_spec.rb
). Note: the data setup might need to be modified to make sure the data is populated - Validate any updates to the new fields are reflected in the index
Edited by Terri Chu