Global Search Group - 15.6 Planning
Priorities for %15.6
Features
Product Manager @JohnMcGuire
In the %15.6, our main focus is to continue working on the features that we started in prior milestones.
-
Add search filtering by programming language.
We have finished the backend development in %15.3. The backend logic is behind a feature flag currently and we plan to enable it globally in %15.5 or %15.6. We will start the frontend in %15.6 -
Add All Users to the Elasticsearch index to make user list faster.
Currently, user information is not indexed in the Advanced Search. User searches still need to get the data from the database. It can be slow and the requests time out quite often. In %15.6, we will start resolving this issue. We will start from the backend logic. -
As part of Document the REST API in OpenAPI V2, we will be working on annotating Global Search APIs.
-
(Stretch) Add Labels as a facet in Advanced Search for Issues.
We started indexing labels for Issues in %15.4. In %15.5, we finished the backend work including Labels as a facet: Aggregations API for labels. In %15.6, we hope to start the frontend development. The frontend work is tracked in the following issue which may get refined when we start working on it.
Maintenance
Engineering Manager: @changzhengliu
-
Index lifecycle management for Advanced Search indices.
As stated in this epic, manual index management does not scale when it comes to an ever-increasing Elasticsearch index. The design of the first iteration can be found at gitlab-org&8746. In %15.5, we started to explore index lifecycle management. The work will continue in %15.6. Please check the following issues for more details and progress of the work -
Global Search Error Budget
As we are finishing the work in Proposal: Error Budget Improvement for Global Search, we will need to watch closely on our Error Budget. Currently, we have two major issues in our mind to look at in the next milestone.- A loading issue on the Global Search Error Budget dashboard
- Add global_search_indexing to sidekiq service
- Add ErrorRate kind to global search SLI library
- (Stretch) We are still overspending our budget at the moment. We need to look into the causes once the work in Proposal: Error Budget Improvement for Global Search is completed
-
Use Rally for Elasticsearch performance testing.
As we were considering what would be our benchmarking toolkit to measure our performance, Rally stood out to be an ideal candidate to perform such tasks. In %15.6, we'd like to look into how to set up a custom Rally track for our benchmarking test for Elasticsearch performance. -
Potential Performance Improvement
The urgency of this effort is driven by the Global Search Error Budget. While we will do root cause analysis on the violation, the following items are in our mind now which could help improve the performance.-
Validate and fix any issues with shard size and document count in Advanced Search indexes.
The documentation states to consider the number of documents in the index when determining shard size. We have identified that at least one index, Notes, did not follow the guidance. So, we will validate this and fix any potential issues.
-
Validate and fix any issues with shard size and document count in Advanced Search indexes.
-
Outline limitations of special character use in search within the Global Search documentation We will look into the first iteration of this SUSImpacting work.
Bugs
Quality Engineer @ebanks
- Global Search should handle files with long lines in code search results without trying to render the whole line
- Global Search Results should not render HTML
- gitlab:elastic:index task does not check pause setting
- ActiveRecord::InvalidForeignKey: PG::ForeignKeyViolation: ERROR: insert or update on table "index_statuses" violates foreign key ...
UX Design
Designer @nickbrandt
- Ability to filter out forks when searching code
- Provide indicator for multiple matches in code results
-
Support multi-project selection for Projects filter
- This one had some previous work done on it, but now that we are moving these filters to the left sidebar it opens up the ability to handle this a bit better (similar to what is shown here)
Kickoff video
CC: @dgruzd @terrichu @john-mason @tbulva @maddievn @sdungarwal