Zoekt Backend for multiple matches in a single file
What does this MR do and why?
What: A new graphql API has been created. This API will call the @results = @search_service.search_objects to fetch the results that we are using for the web as well. Currently, on the web, if the Zoekt is not available for a specific search we fall back to advanced and finally to basic. But in this API, if Zoekt is not available for request, we will raise an error. Frontend will show an error message to the user instead of rendering the results from the advanced or basic search.
In the backend core logic, I have added an extra parameter multi_match_enabled in the initialization of Search::Zoekt::SearchResults. This parameter will decide how to create the results. If the parameter is set to true, the results will consist of a paginated array of Gitlab::Search::FoundMultiLineBlob else it will be Gitlab::Search::FoundBlob as before.
The main logic to create multiline blobs exists in the class Search::Zoekt::MultiMatch.
Why: For better User experience we want to show multiple matches in a single file as one result. For that, we need to create an API to send the data to the frontend so that the frontend can display it accordingly.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
A sample response:
{
"data": {
"blobSearch": {
"matchCount": 34,
"perPage": 20,
"fileCount": 3,
"searchType": "ZOEKT",
"searchLevel": "PROJECT",
"files": [
{
"path": "ee/app/models/search/zoekt/index.rb",
"fileUrl": "http://127.0.0.1:3000/flightjs/Flight/-/blob/master/ee/app/models/search/zoekt/index.rb",
"blameUrl": "http://127.0.0.1:3000/flightjs/Flight/-/blame/master/ee/app/models/search/zoekt/index.rb",
"matchCountTotal": 24,
"matchCount": 12,
"projectPath": "flightjs/Flight",
"chunks": [
{
"matchCountInChunk": 2,
"lines": [
{
"lineNumber": 3,
"richText": "module Search",
"text": "module Search"
},
{
"lineNumber": 4,
"richText": " module <b>Zoekt</b>",
"text": " module Zoekt"
},
{
"lineNumber": 5,
"richText": " class Index < ApplicationRecord",
"text": " class Index < ApplicationRecord"
},
{
"lineNumber": 6,
"richText": " self.table_name = '<b>zoekt</b>_indices'",
"text": " self.table_name = 'zoekt_indices'"
},
{
"lineNumber": 7,
"richText": " include EachBatch",
"text": " include EachBatch"
}
]
},
{
"matchCountInChunk": 9,
"lines": [
{
"lineNumber": 11,
"richText": "",
"text": ""
},
{
"lineNumber": 12,
"richText": " belongs_to :<b>zoekt</b>_enabled_namespace, inverse_of: :indices, class_name: '::Search::<b>Zoekt</b>::EnabledNamespace'",
"text": " belongs_to :zoekt_enabled_namespace, inverse_of: :indices, class_name: '::Search::Zoekt::EnabledNamespace'"
},
{
"lineNumber": 13,
"richText": " belongs_to :node, foreign_key: :<b>zoekt</b>_node_id, inverse_of: :indices, class_name: '::Search::<b>Zoekt</b>::Node'",
"text": " belongs_to :node, foreign_key: :zoekt_node_id, inverse_of: :indices, class_name: '::Search::Zoekt::Node'"
},
{
"lineNumber": 14,
"richText": " belongs_to :replica, foreign_key: :<b>zoekt</b>_replica_id, inverse_of: :indices",
"text": " belongs_to :replica, foreign_key: :zoekt_replica_id, inverse_of: :indices"
},
{
"lineNumber": 15,
"richText": "",
"text": ""
},
{
"lineNumber": 16,
"richText": " has_many :<b>zoekt</b>_repositories, foreign_key: :<b>zoekt</b>_index_id, inverse_of: :<b>zoekt</b>_index,",
"text": " has_many :zoekt_repositories, foreign_key: :zoekt_index_id, inverse_of: :zoekt_index,"
},
{
"lineNumber": 17,
"richText": " class_name: '::Search::<b>Zoekt</b>::Repository'",
"text": " class_name: '::Search::Zoekt::Repository'"
},
{
"lineNumber": 18,
"richText": "",
"text": ""
}
]
},
{
"matchCountInChunk": 1,
"lines": [
{
"lineNumber": 35,
"richText": " scope :for_root_namespace_id, ->(root_namespace_id) do",
"text": " scope :for_root_namespace_id, ->(root_namespace_id) do"
},
{
"lineNumber": 36,
"richText": " where(namespace_id: root_namespace_id).where.not(<b>zoekt</b>_enabled_namespace_id: nil)",
"text": " where(namespace_id: root_namespace_id).where.not(zoekt_enabled_namespace_id: nil)"
},
{
"lineNumber": 37,
"richText": " end",
"text": " end"
}
]
}
]
},
{
"path": "ZoektTest",
"fileUrl": "http://127.0.0.1:3000/flightjs/Flight/-/blob/master/ZoektTest",
"blameUrl": "http://127.0.0.1:3000/flightjs/Flight/-/blame/master/ZoektTest",
"matchCountTotal": 6,
"matchCount": 6,
"projectPath": "flightjs/Flight",
"chunks": [
{
"matchCountInChunk": 2,
"lines": [
{
"lineNumber": 1,
"richText": "# <b>Zoekt</b>Test12 <div> test </div>",
"text": "# ZoektTest12 <div> test </div>"
},
{
"lineNumber": 2,
"richText": "<b>Zoekt</b>Test lucky lucky lucky",
"text": "ZoektTest lucky lucky lucky"
},
{
"lineNumber": 3,
"richText": "lucky",
"text": "lucky"
}
]
},
{
"matchCountInChunk": 4,
"lines": [
{
"lineNumber": 4,
"richText": "",
"text": ""
},
{
"lineNumber": 5,
"richText": "<b>zoekt</b>test123 <div> <b>zoekt</b> </div>",
"text": "zoekttest123 <div> zoekt </div>"
},
{
"lineNumber": 6,
"richText": "",
"text": ""
},
{
"lineNumber": 7,
"richText": "class <b>Zoekt</b>Test",
"text": "class ZoektTest"
},
{
"lineNumber": 8,
"richText": " "<b>zoekt</b>_test.rb"",
"text": " \"zoekt_test.rb\""
},
{
"lineNumber": 9,
"richText": "end",
"text": "end"
}
]
}
]
},
{
"path": "zoekt_test.rb",
"fileUrl": "http://127.0.0.1:3000/flightjs/Flight/-/blob/master/zoekt_test.rb",
"blameUrl": "http://127.0.0.1:3000/flightjs/Flight/-/blame/master/zoekt_test.rb",
"matchCountTotal": 1,
"matchCount": 1,
"projectPath": "flightjs/Flight",
"chunks": [
{
"matchCountInChunk": 1,
"lines": [
{
"lineNumber": 1,
"richText": "class <b>Zoekt</b>Test",
"text": "class ZoektTest"
},
{
"lineNumber": 2,
"richText": " def test",
"text": " def test"
}
]
}
]
}
]
}
}
}| Before | After |
|---|---|
How to set up and validate locally
- Make sure the zoekt is setup.
- Make sure there are no pending db migrations.
- Visit the graphql explorer: http://127.0.0.1:3000/-/graphql-explorer
- Enter the following query. You must replace the groupId and projected which you have chosen to set up zoekt:
query {
blobSearch(
search: "zoekt",
groupId: "gid://gitlab/Group/33",
projectId: "gid://gitlab/Project/7"
) {
matchCount
perPage
fileCount
searchType
searchLevel
files {
path
fileUrl
blameUrl
matchCountTotal
matchCount
projectPath
chunks {
matchCountInChunk
lines {
lineNumber
richText
text
}
}
}
}
}- Check that you get the correct response.
- Remove projectID and groupID from the query. And check that you get an error message:
"Zoekt search is not available for this request" - Now only remove
searchfrom the query. And check that you get an error message:"Field 'blobSearch' is missing required arguments: search" - Now set the
searchto*. And check that you get an error message:"error parsing regexp: missing argument to repetition operator:*" - Now add a new argument in the search query
repositoryRef: "dummy". And check that you get an error message: `"Search is only allowed in project default branch"
Related to #450736 (closed)
Edited by Ravi Kumar