Global Search Group - 17.2 Planning
%17.2
Priorities forGitLab release schedule: https://about.gitlab.com/releases/
Features
Product Manager @bvenker
global searchAI Hybrid search and RAG
Hybrid search was temporarily blocked by having to migrate to using the AI Gateway for embeddings, but with that work completed, we're back on track to release hybrid issues search for a subset of GitLab projects on .com in 17.2.
We're also collaborating with groupcode creation team on RAG solutions for providing code as context for our code-related AI features.
Issue | Estimate |
---|---|
Hybrid issue search results in Advanced Search | 3 |
Experiment with code chunking strategies | spike |
Backfill embeddings for gitlab groups | 3 |
global searchcode search API and scaling infrastructure improvements
Issue | Estimate |
---|---|
Frontend: Add fork filter for Zoekt code search | 3 |
Backend for multiple matches in a single file | 5 |
Spike: zoekt webserver binary into indexer project | spike |
Zoekt: sharding strategy | All issues in linked epic |
global searchadvanced search Improving database performance and developer experience
Merge request filter parity in Advanced Search
Our first phase of moving heavy database queries off of Postgres is bringing the MR filters and searches backed by Postgres into Elasticsearch. In this case, the expected benefit is faster MR text searching and filtering, and potentially the ability to remove large, heavy GIN indexes from Postgres, greatly reducing our WAL issues.
You can follow our overall progress on merge request filter parity in Bring advanced search merge requests search fac... (gitlab-org&13314). We're going to be tackling a few filters per milestone until we're finished, and then we'll collaborate with other teams to get MR search backends ported to these Elasticsearch-powered filters and searches.
Advanced Search framework refactor
We're continuing to make significant improvements to our Advanced Search framework. This is more important than ever as we see more and more GitLab teams wanting to migrate existing features to Elasticsearch and develop new features on top of it.
Issue | Estimate |
---|---|
Merge requests: Search query builder refactor | 5 |
Refactor Search to make calls directly to client | 5 |
Supporting Work Items
We continue to collaborate with groupproject management on the Work Items migration, ensuring that Work Items are indexed and searchable in Advanced Search.
Maintenance
Engineering Manager: @changzhengliu
- Continue with the effort of making Elasticsearch framework easy to use
- Merge requests: Search query builder refactor (gitlab-org/gitlab#462679 - closed). We have a series of issues to simplify the logic of generating Elasticsearch queries for different data types. This work is to simplify the queries for merge requests.
- Refactor Search to make calls directly to client (gitlab-org/gitlab#461874 - closed). It's not straightforward to add and search new type of data on Elasticsearch, and it's especially different for data not modeled in database. This change aims to break this barrier and make queries easier to handle.
- Add search_type parameter to Web and API. With this option, the Code Suggestions could use the API to choose to use Elasticsearch or Zoekt when pulling the search result.
Bugs
Quality Engineering Manager: @ksvoboda
- Investigate TestIndexRepository failure ( gitlab-org/gitlab-zoekt-indexer#43 (closed))
- Advanced search appears to keep PostgreSQL wait... ( gitlab-org/gitlab#465980 (closed))
- User/Project command palette search are sending incorrect scope
- Code search results inconsistent for camel case text
- Zoekt deletion is dependant on elasticsearch setting
UX Design
Technical Writing
Technical Writer: @ashrafkhamis
CC: @dgruzd @terrichu @johnmason @tbulva @maddievn @sdungarwal @rkumar555