Include namespaces in project searches when query contains '/'
What does this MR do and why?
Describe in detail what your merge request does and why.
Currently, most project searches only match against the project's path name, without the group name being included.
For example, using the hamburger menu and searching for gitlab-org/gitlab
won't return the gitlab repo in the results.
This is because the project path is gitlab
and namespaces (gitlab-org/
) aren't included in the search.
This behavior can be controlled using the search_namespaces
parameter. If /
is included in the search query,
then the user is most likely trying to search for a path with a group name. In that case, namespaces should be
included in the search.
Since these queries are issued from the REST API, they are scoped to project authorizations and also paginated, so the performance is decent. To minimize the risk of this causing performance issues in production, a feature flag is used.
SELECT
"projects".*
FROM
"projects"
INNER JOIN "project_authorizations" ON "projects"."id" = "project_authorizations"."project_id"
INNER JOIN "routes" ON "routes"."source_type" = 'Project'
AND "routes"."source_id" = "projects"."id"
WHERE
"project_authorizations"."user_id" = 8953999
AND "projects"."pending_delete" = FALSE
AND (
(
"routes"."path" ILIKE '%gitlab-org/%'
OR "routes"."name" ILIKE '%gitlab-org/%'
)
OR "projects"."description" ILIKE '%gitlab-org/%'
)
AND "projects"."hidden" = FALSE limit 20;
Limit (cost=1066.67..13965.07 rows=2 width=764) (actual time=171.319..710.235 rows=20 loops=1)
Buffers: shared hit=405 read=639 dirtied=18
I/O Timings: read=1665.054 write=0.000
-> Gather (cost=1066.67..13965.07 rows=2 width=764) (actual time=171.317..710.217 rows=20 loops=1)
Workers Planned: 2
Workers Launched: 2
Buffers: shared hit=405 read=639 dirtied=18
I/O Timings: read=1665.054 write=0.000
-> Nested Loop (cost=66.67..12964.87 rows=1 width=764) (actual time=176.331..625.943 rows=7 loops=3)
Buffers: shared hit=405 read=639 dirtied=18
I/O Timings: read=1665.054 write=0.000
-> Nested Loop (cost=66.23..12169.86 rows=1463 width=58) (actual time=131.620..426.889 rows=33 loops=3)
Buffers: shared hit=245 read=397 dirtied=14
I/O Timings: read=1076.893 write=0.000
-> Parallel Bitmap Heap Scan on public.project_authorizations (cost=65.66..6303.79 rows=1645 width=4) (actual time=113.473..212.862 rows=33 loops=3)
Buffers: shared read=140 dirtied=1
I/O Timings: read=462.056 write=0.000
-> Bitmap Index Scan using project_authorizations_pkey (cost=0.00..64.68 rows=3947 width=0) (actual time=108.722..108.723 rows=8043 loops=1)
Index Cond: (project_authorizations.user_id = 8953999)
Buffers: shared read=40
I/O Timings: read=106.079 write=0.000
-> Index Scan using index_routes_on_source_type_and_source_id on public.routes (cost=0.56..3.57 rows=1 width=54) (actual time=6.410..6.410 rows=1 loops=100)
Index Cond: (((routes.source_type)::text = 'Project'::text) AND (routes.source_id = project_authorizations.project_id))
Buffers: shared hit=245 read=257 dirtied=13
I/O Timings: read=614.837 write=0.000
-> Index Scan using projects_pkey on public.projects (cost=0.44..0.53 rows=1 width=764) (actual time=5.962..5.962 rows=0 loops=100)
Index Cond: (projects.id = routes.source_id)
Filter: ((NOT projects.pending_delete) AND (NOT projects.hidden) AND (((routes.path)::text ~~* '%gitlab-org/%'::text) OR ((routes.name)::text ~~* '%gitlab-org/%'::text) OR (projects.description ~~* '%gitlab-org/%'::text)))
Rows Removed by Filter: 1
Buffers: shared hit=160 read=242 dirtied=4
I/O Timings: read=588.161 write=0.000
COLD
Time: 732.639 ms
- planning: 22.165 ms
- execution: 710.474 ms
- I/O read: 1.665 s
- I/O write: 0.000 ms
Shared buffers:
- hits: 405 (~3.20 MiB) from the buffer pool
- reads: 639 (~5.00 MiB) from the OS file cache, including disk I/O
- dirtied: 18 (~144.00 KiB)
- writes: 0
WARM
Time: 28.765 ms
- planning: 14.818 ms
- execution: 13.947 ms
- I/O read: 0.000 ms
- I/O write: 0.000 ms
Shared buffers:
- hits: 1044 (~8.20 MiB) from the buffer pool
- reads: 0 from the OS file cache, including disk I/O
- dirtied: 0
- writes: 0
The cold-cache time exceeds the timing guidelines, but it performs about the same as the old query:
Click to expand
SELECT
"projects".*
FROM
"projects"
INNER JOIN "project_authorizations" ON "projects"."id" = "project_authorizations"."project_id"
WHERE
"project_authorizations"."user_id" = 8953999
AND "projects"."pending_delete" = FALSE
AND (
"projects"."name" ILIKE '%gitlab-org/%'
OR "projects"."path" ILIKE '%gitlab-org/%'
OR "projects"."description" ILIKE '%gitlab-org/%'
)
AND "projects"."hidden" = FALSE limit 20;
Limit (cost=1066.10..12945.52 rows=3 width=764) (actual time=4.558..13.914 rows=20 loops=1)
Buffers: shared hit=2039
I/O Timings: read=0.000 write=0.000
-> Gather (cost=1066.10..12945.52 rows=3 width=764) (actual time=4.556..13.904 rows=20 loops=1)
Workers Planned: 2
Workers Launched: 2
Buffers: shared hit=2039
I/O Timings: read=0.000 write=0.000
-> Nested Loop (cost=66.10..11945.22 rows=1 width=764) (actual time=1.536..3.396 rows=7 loops=3)
Buffers: shared hit=2039
I/O Timings: read=0.000 write=0.000
-> Parallel Bitmap Heap Scan on public.project_authorizations (cost=65.66..6303.79 rows=1645 width=4) (actual time=0.853..1.216 rows=137 loops=3)
Buffers: shared hit=393
I/O Timings: read=0.000 write=0.000
-> Bitmap Index Scan using project_authorizations_pkey (cost=0.00..64.68 rows=3947 width=0) (actual time=1.694..1.695 rows=8043 loops=1)
Index Cond: (project_authorizations.user_id = 8953999)
Buffers: shared hit=40
I/O Timings: read=0.000 write=0.000
-> Index Scan using projects_pkey on public.projects (cost=0.44..3.43 rows=1 width=764) (actual time=0.015..0.015 rows=0 loops=411)
Index Cond: (projects.id = project_authorizations.project_id)
Filter: ((NOT projects.pending_delete) AND (NOT projects.hidden) AND (((projects.name)::text ~~* '%gitlab-org/%'::text) OR ((projects.path)::text ~~* '%gitlab-org/%'::text) OR (projects.description ~~* '%gitlab-org/%'::text)))
Rows Removed by Filter: 1
Buffers: shared hit=1646
I/O Timings: read=0.000 write=0.000
COLD
Time: 959.031 ms
- planning: 5.326 ms
- execution: 953.705 ms
- I/O read: 2.559 s
- I/O write: 0.000 ms
Shared buffers:
- hits: 1353 (~10.60 MiB) from the buffer pool
- reads: 1057 (~8.30 MiB) from the OS file cache, including disk I/O
- dirtied: 47 (~376.00 KiB)
- writes: 0
WARM
Time: 20.596 ms
- planning: 6.509 ms
- execution: 14.087 ms
- I/O read: 0.000 ms
- I/O write: 0.000 ms
Shared buffers:
- hits: 2039 (~15.90 MiB) from the buffer pool
- reads: 0 from the OS file cache, including disk I/O
- dirtied: 0
- writes: 0
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Location | Feature flag disabled | Feature flag enabled |
---|---|---|
Main menu | ||
Import project members | ||
Todo list project filter |
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
- Log in to gdk as root
- Click the hamburger menu in the top left and search for
gitlab-org/
- Receive no results
- Enable the
:full_path_project_search
feature flag using the rails console - Reload the page
- Do the search again
- Receive results
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.