Backend Zoekt for multiple matches in a single file
Create a new class or API endpoint to support grouping matches by file so they can be displayed by file in the UI
- Limit 50 matches per file result (but experiment with performance to see what the optimal number is)
- Frontend will migrate to use Vue instead of HAML
Questions
- @terrichu: Will API results match this approach?
Update
We have decided to implement this in GraphQL. The whole feature should be behind a WIP:
feature flag. If the feature flag is turned on, FE will fetch the results from the backend API and if the FF is turned off the FE will fallback to Rails HAML. In the first iteration, we will implement only the Zoekt search. If a Zoekt is disabled for a specific API request then the API will send 400
response to the FE and the FE will handle it accordingly. Let's say FE will display a message to the user that the Zoekt
is disabled for the search. The FF should be enabled globally only after the full implementation of all three types of search basic
, advanced
, and zoekt
.
Sample request:
curl --location 'http://127.0.0.1:3000/api/graphql' \
--header 'Content-Type: application/json' \
--header 'Accept-Encoding;' \
--header 'Authorization: Bearer glpat-vB2MXZJr6z34fZs*****' \
--data '{"query":"query {\n blobSearch(\n search: \"zoekt\",\n groupId: \"gid://gitlab/Group/33\",\n projectId: \"gid://gitlab/Project/7\"\n ) {\n matchCount\n perPage\n fileCount\n searchType\n searchLevel\n files {\n path\n fileUrl\n blameUrl\n matchCountTotal\n matchCount\n projectPath\n chunks {\n matchCountInChunk\n lines {\n lineNumber\n richText\n text\n }\n }\n }\n }\n}\n","variables":{}}'
Sample response:
{
"data": {
"blobSearch": {
"matchCount": 34,
"perPage": 20,
"fileCount": 3,
"searchType": "ZOEKT",
"searchLevel": "PROJECT",
"files": [
{
"path": "ZoektTest",
"fileUrl": "http://127.0.0.1:3000/flightjs/Flight/-/blob/master/ZoektTest",
"blameUrl": "http://127.0.0.1:3000/flightjs/Flight/-/blame/master/ZoektTest",
"matchCountTotal": 6,
"matchCount": 6,
"projectPath": "flightjs/Flight",
"chunks": [
{
"matchCountInChunk": 2,
"lines": [
{
"lineNumber": 1,
"richText": "# <b>Zoekt</b>Test12 <div> test </div>",
"text": "# ZoektTest12 <div> test </div>"
},
{
"lineNumber": 2,
"richText": "<b>Zoekt</b>Test lucky lucky lucky",
"text": "ZoektTest lucky lucky lucky"
},
{
"lineNumber": 3,
"richText": "lucky",
"text": "lucky"
}
]
},
{
"matchCountInChunk": 4,
"lines": [
{
"lineNumber": 4,
"richText": "",
"text": ""
},
{
"lineNumber": 5,
"richText": "<b>zoekt</b>test123 <div> <b>zoekt</b> </div>",
"text": "zoekttest123 <div> zoekt </div>"
},
{
"lineNumber": 6,
"richText": "",
"text": ""
},
{
"lineNumber": 7,
"richText": "class <b>Zoekt</b>Test",
"text": "class ZoektTest"
},
{
"lineNumber": 8,
"richText": " "<b>zoekt</b>_test.rb"",
"text": " \"zoekt_test.rb\""
},
{
"lineNumber": 9,
"richText": "end",
"text": "end"
}
]
}
]
},
{
"path": "ee/app/models/search/zoekt/index.rb",
"fileUrl": "http://127.0.0.1:3000/flightjs/Flight/-/blob/master/ee/app/models/search/zoekt/index.rb",
"blameUrl": "http://127.0.0.1:3000/flightjs/Flight/-/blame/master/ee/app/models/search/zoekt/index.rb",
"matchCountTotal": 24,
"matchCount": 12,
"projectPath": "flightjs/Flight",
"chunks": [
{
"matchCountInChunk": 2,
"lines": [
{
"lineNumber": 3,
"richText": "module Search",
"text": "module Search"
},
{
"lineNumber": 4,
"richText": " module <b>Zoekt</b>",
"text": " module Zoekt"
},
{
"lineNumber": 5,
"richText": " class Index < ApplicationRecord",
"text": " class Index < ApplicationRecord"
},
{
"lineNumber": 6,
"richText": " self.table_name = '<b>zoekt</b>_indices'",
"text": " self.table_name = 'zoekt_indices'"
},
{
"lineNumber": 7,
"richText": " include EachBatch",
"text": " include EachBatch"
}
]
},
{
"matchCountInChunk": 9,
"lines": [
{
"lineNumber": 11,
"richText": "",
"text": ""
},
{
"lineNumber": 12,
"richText": " belongs_to :<b>zoekt</b>_enabled_namespace, inverse_of: :indices, class_name: '::Search::<b>Zoekt</b>::EnabledNamespace'",
"text": " belongs_to :zoekt_enabled_namespace, inverse_of: :indices, class_name: '::Search::Zoekt::EnabledNamespace'"
},
{
"lineNumber": 13,
"richText": " belongs_to :node, foreign_key: :<b>zoekt</b>_node_id, inverse_of: :indices, class_name: '::Search::<b>Zoekt</b>::Node'",
"text": " belongs_to :node, foreign_key: :zoekt_node_id, inverse_of: :indices, class_name: '::Search::Zoekt::Node'"
},
{
"lineNumber": 14,
"richText": " belongs_to :replica, foreign_key: :<b>zoekt</b>_replica_id, inverse_of: :indices",
"text": " belongs_to :replica, foreign_key: :zoekt_replica_id, inverse_of: :indices"
},
{
"lineNumber": 15,
"richText": "",
"text": ""
},
{
"lineNumber": 16,
"richText": " has_many :<b>zoekt</b>_repositories, foreign_key: :<b>zoekt</b>_index_id, inverse_of: :<b>zoekt</b>_index,",
"text": " has_many :zoekt_repositories, foreign_key: :zoekt_index_id, inverse_of: :zoekt_index,"
},
{
"lineNumber": 17,
"richText": " class_name: '::Search::<b>Zoekt</b>::Repository'",
"text": " class_name: '::Search::Zoekt::Repository'"
},
{
"lineNumber": 18,
"richText": "",
"text": ""
}
]
},
{
"matchCountInChunk": 1,
"lines": [
{
"lineNumber": 35,
"richText": " scope :for_root_namespace_id, ->(root_namespace_id) do",
"text": " scope :for_root_namespace_id, ->(root_namespace_id) do"
},
{
"lineNumber": 36,
"richText": " where(namespace_id: root_namespace_id).where.not(<b>zoekt</b>_enabled_namespace_id: nil)",
"text": " where(namespace_id: root_namespace_id).where.not(zoekt_enabled_namespace_id: nil)"
},
{
"lineNumber": 37,
"richText": " end",
"text": " end"
}
]
}
]
},
{
"path": "zoekt_test.rb",
"fileUrl": "http://127.0.0.1:3000/flightjs/Flight/-/blob/master/zoekt_test.rb",
"blameUrl": "http://127.0.0.1:3000/flightjs/Flight/-/blame/master/zoekt_test.rb",
"matchCountTotal": 1,
"matchCount": 1,
"projectPath": "flightjs/Flight",
"chunks": [
{
"matchCountInChunk": 1,
"lines": [
{
"lineNumber": 1,
"richText": "class <b>Zoekt</b>Test",
"text": "class ZoektTest"
},
{
"lineNumber": 2,
"richText": " def test",
"text": " def test"
}
]
}
]
}
]
}
}
}
Request attributes:
Field name | Explanation | Required | Default | Example |
---|---|---|---|---|
search | Search term | Yes | foobar | |
groupId | Group to be searched in | No | gid://gitlab/Group/33 | |
projectId | Group to be searched in | No | gid://gitlab/Project/7 | |
page | Page number in the paginated result | No | 1 | 2 |
per_page | Number of files in response | No | 20 | 25 |
repository_ref | repository Ref to be searched in the | No | master | |
chunk_count | Number of chunks per file to be included in a response | No | 3 | 50 |
Response attributes:
Field name | Explanation |
---|---|
blobSearch.matchCount | Total match count in a search |
blobSearch.perPage | Total files per page |
blobSearch.fileCount | Total number of files where there are matches |
blobSearch.searchType | Type of search(ZOEKT) |
blobSearch.searchLevel | Level of search(PROJECT, GROUP, GLOBAL) |
blobSearch.files[].path | File path |
blobSearch.files[].fileUrl | File url |
blobSearch.files[].blameUrl | Blame url |
blobSearch.files[].matchCountTotal | Total matches in a file |
blobSearch.files[].matchCount | Total of all the matches in the requested chunks |
blobSearch.files[].projectPath | Project full path |
blobSearch.files[].chunks[].matchCountInChunk | Total matches in a chunk |
blobSearch.files[].chunks[].lines[].lineNumber | Line number of the blob |
blobSearch.files[].chunks[].lines[].richText | Rich text of the blob |
blobSearch.files[].chunks[].lines[].text | Text of the blob |
Sample error response:
{
"errors": [
{
"message": "error parsing regexp: missing argument to repetition operator: `*`",
"locations": [
{
"line": 2,
"column": 3
}
],
"path": [
"blobSearch"
]
}
],
"data": {
"blobSearch": null
}
}