Pass cross projects artifacts dependencies in CI
What does this MR do?
Expand needs keyword to specify dependency on another project's artifacts. Only five dependencies are allowed.
related issue: #14311 (closed)
image: 'busybox:latest'
build-1:
stage: build
script:
- ls -lhR
needs:
- project: cross-deps/project-1
ref: master
job: build-1
artifacts: true
- project: cross-deps/project-2
ref: master
job: build-2
artifacts: true
- project: cross-deps/project-3
ref: feature
job: build-2
artifacts: trueGenerates a Ci::Build with the following options:
{
"image": {
"name": "busybox:latest"
},
"cross_dependencies": [
{
"project": "cross-deps/project-1",
"ref": "master",
"job": "build-1",
"artifacts": true
},
{
"project": "cross-deps/project-2",
"ref": "master",
"job": "build-2",
"artifacts": true
},
{
"project": "cross-deps/project-3",
"ref": "feature",
"job": "build-2",
"artifacts": true
}
],
"script": [
"ls -lhR"
]
}
The cross_dependencies builds are included with the regular dependencies in the response message for the job /requests made by the runner.
Database queries
To find all the builds specified in the GitLab CI YAML file, only one query will be executed. YAML example:
test:
stage: test
script:
- ls -lhR
needs:
- project: gitlab-org/gitlab
ref: cross-projects-artifacts-dependencies
job: code_quality
artifacts: true
- project: gitlab-org/gitlab
ref: qa-shl-fix-ip-address-whitelisting-e2e-spec
job: dast
artifacts: true
- project: gitlab-org/gitlab
ref: update-page-object-and-view-to-use-data-qa-selector
job: jest
artifacts: true
- project: gitlab-org/gitlab
ref: 36776-entropy-requirements-for-new-user-passwords-mvc
job: coverage
artifacts: true
- project: gitlab-org/gitlab
ref: 31034-cache-classes-in-ci
job: dependency_scanning
artifacts: trueSQL query for this example:
SELECT "ci_builds".* FROM ((SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."id" IN (SELECT max(ci_builds.id) as id FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND ("ci_builds"."retried" = FALSE OR "ci_builds"."retried" IS NULL) AND ("ci_builds"."status" IN ('success')) AND "ci_builds"."name" = 'code_quality' AND "ci_builds"."ref" = 'cross-projects-artifacts-dependencies' AND "ci_builds"."project_id" IN (SELECT "projects"."id" FROM "projects" LEFT OUTER JOIN "routes" ON "routes"."source_id" = "projects"."id" AND "routes"."source_type" = 'Project' WHERE ((LOWER(routes.path) = LOWER('gitlab-org/gitlab'))))))
UNION
(SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."id" IN (SELECT max(ci_builds.id) as id FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND ("ci_builds"."retried" = FALSE OR "ci_builds"."retried" IS NULL) AND ("ci_builds"."status" IN ('success')) AND "ci_builds"."name" = 'dast' AND "ci_builds"."ref" = 'qa-shl-fix-ip-address-whitelisting-e2e-spec' AND "ci_builds"."project_id" IN (SELECT "projects"."id" FROM "projects" LEFT OUTER JOIN "routes" ON "routes"."source_id" = "projects"."id" AND "routes"."source_type" = 'Project' WHERE ((LOWER(routes.path) = LOWER('gitlab-org/gitlab'))))))
UNION
(SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."id" IN (SELECT max(ci_builds.id) as id FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND ("ci_builds"."retried" = FALSE OR "ci_builds"."retried" IS NULL) AND ("ci_builds"."status" IN ('success')) AND "ci_builds"."name" = 'jest' AND "ci_builds"."ref" = 'update-page-object-and-view-to-use-data-qa-selector' AND "ci_builds"."project_id" IN (SELECT "projects"."id" FROM "projects" LEFT OUTER JOIN "routes" ON "routes"."source_id" = "projects"."id" AND "routes"."source_type" = 'Project' WHERE ((LOWER(routes.path) = LOWER('gitlab-org/gitlab'))))))
UNION
(SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."id" IN (SELECT max(ci_builds.id) as id FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND ("ci_builds"."retried" = FALSE OR "ci_builds"."retried" IS NULL) AND ("ci_builds"."status" IN ('success')) AND "ci_builds"."name" = 'coverage' AND "ci_builds"."ref" = '36776-entropy-requirements-for-new-user-passwords-mvc' AND "ci_builds"."project_id" IN (SELECT "projects"."id" FROM "projects" LEFT OUTER JOIN "routes" ON "routes"."source_id" = "projects"."id" AND "routes"."source_type" = 'Project' WHERE ((LOWER(routes.path) = LOWER('gitlab-org/gitlab'))))))
UNION
(SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."id" IN (SELECT max(ci_builds.id) as id FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND ("ci_builds"."retried" = FALSE OR "ci_builds"."retried" IS NULL) AND ("ci_builds"."status" IN ('success')) AND "ci_builds"."name" = 'dependency_scanning' AND "ci_builds"."ref" = '31034-cache-classes-in-ci' AND "ci_builds"."project_id" IN (SELECT "projects"."id" FROM "projects" LEFT OUTER JOIN "routes" ON "routes"."source_id" = "projects"."id" AND "routes"."source_type" = 'Project' WHERE ((LOWER(routes.path) = LOWER('gitlab-org/gitlab'))))))) ci_builds WHERE "ci_builds"."type" IN ('Ci::Build')database-lab explanation for this query without additional indices:
Plan without execution
HashAggregate (cost=3712.92..3712.97 rows=5 width=4108)
Group Key: ci_builds.id, ci_builds.status, ci_builds.finished_at, ci_builds.trace, ci_builds.created_at, ci_builds.updated_at, ci_builds.started_at, ci_builds.runner_id, ci_builds.coverage, ci_builds.commit_id, ci_builds.commands, ci_builds.name, ci_builds.options, ci_builds.allow_failure, ci_builds.stage, ci_builds.trigger_request_id, ci_builds.stage_idx, ci_builds.tag, ci_builds.ref, ci_builds.user_id, ci_builds.type, ci_builds.target_url, ci_builds.description, ci_builds.artifacts_file, ci_builds.project_id, ci_builds.artifacts_metadata, ci_builds.erased_by_id, ci_builds.erased_at, ci_builds.artifacts_expire_at, ci_builds.environment, ci_builds.artifacts_size, ci_builds."when", ci_builds.yaml_variables, ci_builds.queued_at, ci_builds.token, ci_builds.lock_version, ci_builds.coverage_regex, ci_builds.auto_canceled_by_id, ci_builds.retried, ci_builds.stage_id, ci_builds.artifacts_file_store, ci_builds.artifacts_metadata_store, ci_builds.protected, ci_builds.failure_reason, ci_builds.scheduled_at, ci_builds.token_encrypted, ci_builds.upstream_pipeline_id
-> Append (cost=738.41..3712.33 rows=5 width=4108)
-> Nested Loop (cost=738.41..742.46 rows=1 width=1504)
-> Aggregate (cost=737.83..737.84 rows=1 width=4)
-> Nested Loop (cost=9.62..737.83 rows=1 width=4)
Join Filter: (ci_builds_1.project_id = routes.source_id)
-> HashAggregate (cost=9.05..9.06 rows=1 width=8)
Group Key: projects.id
-> Nested Loop (cost=1.00..9.05 rows=1 width=8)
-> Index Scan using index_on_routes_lower_path on routes (cost=0.56..4.58 rows=1 width=4)
Index Cond: (lower((path)::text) = 'gitlab-org/gitlab'::text)
Filter: ((source_type)::text = 'Project'::text)
-> Index Only Scan using projects_pkey on projects (cost=0.43..4.45 rows=1 width=4)
Index Cond: (id = routes.source_id)
-> Index Scan using index_ci_builds_on_project_id_and_id on ci_builds ci_builds_1 (cost=0.57..728.76 rows=1 width=8)
Index Cond: (project_id = projects.id)
Filter: (((NOT retried) OR (retried IS NULL)) AND ((type)::text = 'Ci::Build'::text) AND ((status)::text = 'success'::text) AND ((name)::text = 'code_quality'::text) AND ((ref)::text = 'cross-projects-artifacts-dependencies'::text))
-> Index Scan using ci_builds_pkey on ci_builds (cost=0.57..4.59 rows=1 width=1504)
Index Cond: (id = (max(ci_builds_1.id)))
Filter: ((type)::text = 'Ci::Build'::text)
-> Nested Loop (cost=738.41..742.46 rows=1 width=1504)
-> Aggregate (cost=737.83..737.84 rows=1 width=4)
-> Nested Loop (cost=9.62..737.83 rows=1 width=4)
Join Filter: (ci_builds_3.project_id = routes_1.source_id)
-> HashAggregate (cost=9.05..9.06 rows=1 width=8)
Group Key: projects_1.id
-> Nested Loop (cost=1.00..9.05 rows=1 width=8)
-> Index Scan using index_on_routes_lower_path on routes routes_1 (cost=0.56..4.58 rows=1 width=4)
Index Cond: (lower((path)::text) = 'gitlab-org/gitlab'::text)
Filter: ((source_type)::text = 'Project'::text)
-> Index Only Scan using projects_pkey on projects projects_1 (cost=0.43..4.45 rows=1 width=4)
Index Cond: (id = routes_1.source_id)
-> Index Scan using index_ci_builds_on_project_id_and_id on ci_builds ci_builds_3 (cost=0.57..728.76 rows=1 width=8)
Index Cond: (project_id = projects_1.id)
Filter: (((NOT retried) OR (retried IS NULL)) AND ((type)::text = 'Ci::Build'::text) AND ((status)::text = 'success'::text) AND ((name)::text = 'dast'::text) AND ((ref)::text = 'qa-shl-fix-ip-address-whitelisting-e2e-spec'::text))
-> Index Scan using ci_builds_pkey on ci_builds ci_builds_2 (cost=0.57..4.59 rows=1 width=1504)
Index Cond: (id = (max(ci_builds_3.id)))
Filter: ((type)::text = 'Ci::Build'::text)
-> Nested Loop (cost=738.41..742.46 rows=1 width=1504)
-> Aggregate (cost=737.83..737.84 rows=1 width=4)
-> Nested Loop (cost=9.62..737.83 rows=1 width=4)
Join Filter: (ci_builds_5.project_id = routes_2.source_id)
-> HashAggregate (cost=9.05..9.06 rows=1 width=8)
Group Key: projects_2.id
-> Nested Loop (cost=1.00..9.05 rows=1 width=8)
-> Index Scan using index_on_routes_lower_path on routes routes_2 (cost=0.56..4.58 rows=1 width=4)
Index Cond: (lower((path)::text) = 'gitlab-org/gitlab'::text)
Filter: ((source_type)::text = 'Project'::text)
-> Index Only Scan using projects_pkey on projects projects_2 (cost=0.43..4.45 rows=1 width=4)
Index Cond: (id = routes_2.source_id)
-> Index Scan using index_ci_builds_on_project_id_and_id on ci_builds ci_builds_5 (cost=0.57..728.76 rows=1 width=8)
Index Cond: (project_id = projects_2.id)
Filter: (((NOT retried) OR (retried IS NULL)) AND ((type)::text = 'Ci::Build'::text) AND ((status)::text = 'success'::text) AND ((name)::text = 'jest'::text) AND ((ref)::text = 'update-page-object-and-view-to-use-data-qa-selector'::text))
-> Index Scan using ci_builds_pkey on ci_builds ci_builds_4 (cost=0.57..4.59 rows=1 width=1504)
Index Cond: (id = (max(ci_builds_5.id)))
Filter: ((type)::text = 'Ci::Build'::text)
-> Nested Loop (cost=738.41..742.46 rows=1 width=1504)
-> Aggregate (cost=737.83..737.84 rows=1 width=4)
-> Nested Loop (cost=9.62..737.83 rows=1 width=4)
Join Filter: (ci_builds_7.project_id = routes_3.source_id)
-> HashAggregate (cost=9.05..9.06 rows=1 width=8)
Group Key: projects_3.id
-> Nested Loop (cost=1.00..9.05 rows=1 width=8)
-> Index Scan using index_on_routes_lower_path on routes routes_3 (cost=0.56..4.58 rows=1 width=4)
Index Cond: (lower((path)::text) = 'gitlab-org/gitlab'::text)
Filter: ((source_type)::text = 'Project'::text)
-> Index Only Scan using projects_pkey on projects projects_3 (cost=0.43..4.45 rows=1 width=4)
Index Cond: (id = routes_3.source_id)
-> Index Scan using index_ci_builds_on_project_id_and_id on ci_builds ci_builds_7 (cost=0.57..728.76 rows=1 width=8)
Index Cond: (project_id = projects_3.id)
Filter: (((NOT retried) OR (retried IS NULL)) AND ((type)::text = 'Ci::Build'::text) AND ((status)::text = 'success'::text) AND ((name)::text = 'coverage'::text) AND ((ref)::text = '36776-entropy-requirements-for-new-user-passwords-mvc'::text))
-> Index Scan using ci_builds_pkey on ci_builds ci_builds_6 (cost=0.57..4.59 rows=1 width=1504)
Index Cond: (id = (max(ci_builds_7.id)))
Filter: ((type)::text = 'Ci::Build'::text)
-> Nested Loop (cost=738.41..742.46 rows=1 width=1504)
-> Aggregate (cost=737.83..737.84 rows=1 width=4)
-> Nested Loop (cost=9.62..737.83 rows=1 width=4)
Join Filter: (ci_builds_9.project_id = routes_4.source_id)
-> HashAggregate (cost=9.05..9.06 rows=1 width=8)
Group Key: projects_4.id
-> Nested Loop (cost=1.00..9.05 rows=1 width=8)
-> Index Scan using index_on_routes_lower_path on routes routes_4 (cost=0.56..4.58 rows=1 width=4)
Index Cond: (lower((path)::text) = 'gitlab-org/gitlab'::text)
Filter: ((source_type)::text = 'Project'::text)
-> Index Only Scan using projects_pkey on projects projects_4 (cost=0.43..4.45 rows=1 width=4)
Index Cond: (id = routes_4.source_id)
-> Index Scan using index_ci_builds_on_project_id_and_id on ci_builds ci_builds_9 (cost=0.57..728.76 rows=1 width=8)
Index Cond: (project_id = projects_4.id)
Filter: (((NOT retried) OR (retried IS NULL)) AND ((type)::text = 'Ci::Build'::text) AND ((status)::text = 'success'::text) AND ((name)::text = 'dependency_scanning'::text) AND ((ref)::text = '31034-cache-classes-in-ci'::text))
-> Index Scan using ci_builds_pkey on ci_builds ci_builds_8 (cost=0.57..4.59 rows=1 width=1504)
Index Cond: (id = (max(ci_builds_9.id)))
Filter: ((type)::text = 'Ci::Build'::text)This query ran for about 4:30 hours before the session was killed, so there is no way to run the full query with the current indices.
Running just the first query fragment from the union in database-lab:
SELECT "ci_builds".*
FROM "ci_builds"
WHERE "ci_builds"."type" IN ('Ci::Build')
AND "ci_builds"."id" IN (
SELECT max(ci_builds.id) as id
FROM "ci_builds"
WHERE "ci_builds"."type" IN ('Ci::Build')
AND ("ci_builds"."retried" = FALSE OR "ci_builds"."retried" IS NULL)
AND ("ci_builds"."status" IN ('success'))
AND "ci_builds"."name" = 'code_quality'
AND "ci_builds"."ref" = 'cross-projects-artifacts-dependencies'
AND "ci_builds"."project_id" IN (
SELECT "projects"."id"
FROM "projects" LEFT OUTER JOIN "routes" ON "routes"."source_id" = "projects"."id" AND "routes"."source_type" = 'Project'
WHERE ((LOWER(routes.path) = LOWER('gitlab-org/gitlab')))))
Time: 60.050 min
- planning: 28.347 ms
- execution: 60.049 min
- I/O read: 41.280 min
- I/O write: 0.000 ms
Shared buffers:
- hits: 6750625 (~51.50 GiB) from the buffer pool
- reads: 7169845 (~54.70 GiB) from the OS file cache, including disk I/O
- dirtied: 8033 (~62.80 MiB)
- writes: 0Full execution plan
Nested Loop (cost=738.41..742.46 rows=1 width=1504) (actual time=3602958.303..3602958.303 rows=0 loops=1)
Buffers: shared hit=6750625 read=7169845 dirtied=8033
I/O Timings: read=2476776.011
-> Aggregate (cost=737.83..737.84 rows=1 width=4) (actual time=3602958.237..3602958.237 rows=1 loops=1)
Buffers: shared hit=6750625 read=7169845 dirtied=8033
I/O Timings: read=2476776.011
-> Nested Loop (cost=9.62..737.83 rows=1 width=4) (actual time=3602958.232..3602958.232 rows=0 loops=1)
Buffers: shared hit=6750625 read=7169845 dirtied=8033
I/O Timings: read=2476776.011
-> HashAggregate (cost=9.05..9.06 rows=1 width=8) (actual time=20.091..20.138 rows=1 loops=1)
Group Key: projects.id
Buffers: shared read=10
I/O Timings: read=19.876
-> Nested Loop (cost=1.00..9.05 rows=1 width=8) (actual time=20.076..20.081 rows=1 loops=1)
Buffers: shared read=10
I/O Timings: read=19.876
-> Index Scan using index_on_routes_lower_path on public.routes (cost=0.56..4.58 rows=1 width=4) (actual time=11.363..11.365 rows=1 loops=1)
Index Cond: (lower((routes.path)::text) = 'gitlab-org/gitlab'::text)
Filter: ((routes.source_type)::text = 'Project'::text)
Rows Removed by Filter: 0
Buffers: shared read=5
I/O Timings: read=11.264
-> Index Only Scan using projects_pkey on public.projects (cost=0.43..4.45 rows=1 width=4) (actual time=8.691..8.693 rows=1 loops=1)
Index Cond: (projects.id = routes.source_id)
Heap Fetches: 1
Buffers: shared read=5
I/O Timings: read=8.612
-> Index Scan using index_ci_builds_on_project_id_and_id on public.ci_builds ci_builds_1 (cost=0.57..728.76 rows=1 width=8) (actual time=3602938.051..3602938.051 rows=0 loops=1)
Index Cond: (ci_builds_1.project_id = projects.id)
Filter: (((NOT ci_builds_1.retried) OR (ci_builds_1.retried IS NULL)) AND ((ci_builds_1.type)::text = 'Ci::Build'::text) AND ((ci_builds_1.status)::text = 'success'::text) AND ((ci_builds_1.name)::text = 'code_quality'::text) AND ((ci_builds_1.ref)::text = 'cross-projects-artifacts-dependencies'::text))
Rows Removed by Filter: 14324153
Buffers: shared hit=6750625 read=7169835 dirtied=8033
I/O Timings: read=2476756.135
-> Index Scan using ci_builds_pkey on public.ci_builds (cost=0.57..4.59 rows=1 width=1504) (actual time=0.002..0.002 rows=0 loops=1)
Index Cond: (ci_builds.id = (max(ci_builds_1.id)))
Filter: ((ci_builds.type)::text = 'Ci::Build'::text)
Rows Removed by Filter: 0After the index was added:
Index execution details: Duration: 101.423 min
Full execution plan for the entire query
HashAggregate (cost=83.50..83.55 rows=5 width=4108) (actual time=54.646..54.646 rows=0 loops=1)
Group Key: ci_builds.id, ci_builds.status, ci_builds.finished_at, ci_builds.trace, ci_builds.created_at, ci_builds.updated_at, ci_builds.started_at, ci_builds.runner_id, ci_builds.coverage, ci_builds.commit_id, ci_builds.commands, ci_builds.name, ci_builds.options, ci_builds.allow_failure, ci_builds.stage, ci_builds.trigger_request_id, ci_builds.stage_idx, ci_builds.tag, ci_builds.ref, ci_builds.user_id, ci_builds.type, ci_builds.target_url, ci_builds.description, ci_builds.artifacts_file, ci_builds.project_id, ci_builds.artifacts_metadata, ci_builds.erased_by_id, ci_builds.erased_at, ci_builds.artifacts_expire_at, ci_builds.environment, ci_builds.artifacts_size, ci_builds."when", ci_builds.yaml_variables, ci_builds.queued_at, ci_builds.token, ci_builds.lock_version, ci_builds.coverage_regex, ci_builds.auto_canceled_by_id, ci_builds.retried, ci_builds.stage_id, ci_builds.artifacts_file_store, ci_builds.artifacts_metadata_store, ci_builds.protected, ci_builds.failure_reason, ci_builds.scheduled_at, ci_builds.token_encrypted, ci_builds.upstream_pipeline_id
Buffers: shared hit=50 read=27 dirtied=3
I/O Timings: read=53.708
-> Append (cost=12.52..82.91 rows=5 width=4108) (actual time=54.644..54.644 rows=0 loops=1)
Buffers: shared hit=50 read=27 dirtied=3
I/O Timings: read=53.708
-> Nested Loop (cost=12.52..16.57 rows=1 width=1504) (actual time=38.166..38.166 rows=0 loops=1)
Buffers: shared read=17 dirtied=3
I/O Timings: read=37.632
-> Aggregate (cost=11.95..11.96 rows=1 width=4) (actual time=38.158..38.159 rows=1 loops=1)
Buffers: shared read=17 dirtied=3
I/O Timings: read=37.632
-> Nested Loop (cost=9.74..11.95 rows=1 width=4) (actual time=38.155..38.155 rows=0 loops=1)
Buffers: shared read=17 dirtied=3
I/O Timings: read=37.632
-> HashAggregate (cost=9.05..9.06 rows=1 width=8) (actual time=26.187..26.188 rows=1 loops=1)
Group Key: projects.id
Buffers: shared read=12 dirtied=3
I/O Timings: read=25.772
-> Nested Loop (cost=1.00..9.05 rows=1 width=8) (actual time=22.863..26.177 rows=1 loops=1)
Buffers: shared read=12 dirtied=3
I/O Timings: read=25.772
-> Index Scan using index_on_routes_lower_path on public.routes (cost=0.56..4.58 rows=1 width=4) (actual time=14.432..14.434 rows=1 loops=1)
Index Cond: (lower((routes.path)::text) = 'gitlab-org/gitlab'::text)
Filter: ((routes.source_type)::text = 'Project'::text)
Rows Removed by Filter: 0
Buffers: shared read=5
I/O Timings: read=14.301
-> Index Only Scan using projects_pkey on public.projects (cost=0.43..4.45 rows=1 width=4) (actual time=8.411..11.720 rows=1 loops=1)
Index Cond: (projects.id = routes.source_id)
Heap Fetches: 2
Buffers: shared read=7 dirtied=3
I/O Timings: read=11.471
-> Index Scan using index_cross_projects_artifacts on public.ci_builds ci_builds_1 (cost=0.70..2.88 rows=1 width=8) (actual time=11.962..11.962 rows=0 loops=1)
Index Cond: ((ci_builds_1.project_id = projects.id) AND ((ci_builds_1.name)::text = 'code_quality'::text) AND ((ci_builds_1.ref)::text = 'cross-projects-artifacts-dependencies'::text))
Buffers: shared read=5
I/O Timings: read=11.860
-> Index Scan using ci_builds_pkey on public.ci_builds (cost=0.57..4.59 rows=1 width=1504) (actual time=0.001..0.001 rows=0 loops=1)
Index Cond: (ci_builds.id = (max(ci_builds_1.id)))
Filter: ((ci_builds.type)::text = 'Ci::Build'::text)
Rows Removed by Filter: 0
-> Nested Loop (cost=12.52..16.57 rows=1 width=1504) (actual time=3.279..3.279 rows=0 loops=1)
Buffers: shared hit=13 read=2
I/O Timings: read=3.158
-> Aggregate (cost=11.95..11.96 rows=1 width=4) (actual time=3.274..3.274 rows=1 loops=1)
Buffers: shared hit=13 read=2
I/O Timings: read=3.158
-> Nested Loop (cost=9.74..11.95 rows=1 width=4) (actual time=3.271..3.271 rows=0 loops=1)
Buffers: shared hit=13 read=2
I/O Timings: read=3.158
-> HashAggregate (cost=9.05..9.06 rows=1 width=8) (actual time=0.061..0.062 rows=1 loops=1)
Group Key: projects_1.id
Buffers: shared hit=10
-> Nested Loop (cost=1.00..9.05 rows=1 width=8) (actual time=0.054..0.055 rows=1 loops=1)
Buffers: shared hit=10
-> Index Scan using index_on_routes_lower_path on public.routes routes_1 (cost=0.56..4.58 rows=1 width=4) (actual time=0.031..0.032 rows=1 loops=1)
Index Cond: (lower((routes_1.path)::text) = 'gitlab-org/gitlab'::text)
Filter: ((routes_1.source_type)::text = 'Project'::text)
Rows Removed by Filter: 0
Buffers: shared hit=5
-> Index Only Scan using projects_pkey on public.projects projects_1 (cost=0.43..4.45 rows=1 width=4) (actual time=0.013..0.013 rows=1 loops=1)
Index Cond: (projects_1.id = routes_1.source_id)
Heap Fetches: 1
Buffers: shared hit=5
-> Index Scan using index_cross_projects_artifacts on public.ci_builds ci_builds_3 (cost=0.70..2.88 rows=1 width=8) (actual time=3.208..3.208 rows=0 loops=1)
Index Cond: ((ci_builds_3.project_id = projects_1.id) AND ((ci_builds_3.name)::text = 'dast'::text) AND ((ci_builds_3.ref)::text = 'qa-shl-fix-ip-address-whitelisting-e2e-spec'::text))
Buffers: shared hit=3 read=2
I/O Timings: read=3.158
-> Index Scan using ci_builds_pkey on public.ci_builds ci_builds_2 (cost=0.57..4.59 rows=1 width=1504) (actual time=0.001..0.001 rows=0 loops=1)
Index Cond: (ci_builds_2.id = (max(ci_builds_3.id)))
Filter: ((ci_builds_2.type)::text = 'Ci::Build'::text)
Rows Removed by Filter: 0
-> Nested Loop (cost=12.52..16.57 rows=1 width=1504) (actual time=5.472..5.472 rows=0 loops=1)
Buffers: shared hit=12 read=3
I/O Timings: read=5.379
-> Aggregate (cost=11.95..11.96 rows=1 width=4) (actual time=5.467..5.467 rows=1 loops=1)
Buffers: shared hit=12 read=3
I/O Timings: read=5.379
-> Nested Loop (cost=9.74..11.95 rows=1 width=4) (actual time=5.464..5.464 rows=0 loops=1)
Buffers: shared hit=12 read=3
I/O Timings: read=5.379
-> HashAggregate (cost=9.05..9.06 rows=1 width=8) (actual time=0.043..0.044 rows=1 loops=1)
Group Key: projects_2.id
Buffers: shared hit=10
-> Nested Loop (cost=1.00..9.05 rows=1 width=8) (actual time=0.034..0.035 rows=1 loops=1)
Buffers: shared hit=10
-> Index Scan using index_on_routes_lower_path on public.routes routes_2 (cost=0.56..4.58 rows=1 width=4) (actual time=0.019..0.019 rows=1 loops=1)
Index Cond: (lower((routes_2.path)::text) = 'gitlab-org/gitlab'::text)
Filter: ((routes_2.source_type)::text = 'Project'::text)
Rows Removed by Filter: 0
Buffers: shared hit=5
-> Index Only Scan using projects_pkey on public.projects projects_2 (cost=0.43..4.45 rows=1 width=4) (actual time=0.007..0.008 rows=1 loops=1)
Index Cond: (projects_2.id = routes_2.source_id)
Heap Fetches: 1
Buffers: shared hit=5
-> Index Scan using index_cross_projects_artifacts on public.ci_builds ci_builds_5 (cost=0.70..2.88 rows=1 width=8) (actual time=5.417..5.417 rows=0 loops=1)
Index Cond: ((ci_builds_5.project_id = projects_2.id) AND ((ci_builds_5.name)::text = 'jest'::text) AND ((ci_builds_5.ref)::text = 'update-page-object-and-view-to-use-data-qa-selector'::text))
Buffers: shared hit=2 read=3
I/O Timings: read=5.379
-> Index Scan using ci_builds_pkey on public.ci_builds ci_builds_4 (cost=0.57..4.59 rows=1 width=1504) (actual time=0.001..0.001 rows=0 loops=1)
Index Cond: (ci_builds_4.id = (max(ci_builds_5.id)))
Filter: ((ci_builds_4.type)::text = 'Ci::Build'::text)
Rows Removed by Filter: 0
-> Nested Loop (cost=12.52..16.57 rows=1 width=1504) (actual time=2.278..2.278 rows=0 loops=1)
Buffers: shared hit=13 read=2
I/O Timings: read=2.187
-> Aggregate (cost=11.95..11.96 rows=1 width=4) (actual time=2.273..2.273 rows=1 loops=1)
Buffers: shared hit=13 read=2
I/O Timings: read=2.187
-> Nested Loop (cost=9.74..11.95 rows=1 width=4) (actual time=2.270..2.270 rows=0 loops=1)
Buffers: shared hit=13 read=2
I/O Timings: read=2.187
-> HashAggregate (cost=9.05..9.06 rows=1 width=8) (actual time=0.048..0.049 rows=1 loops=1)
Group Key: projects_3.id
Buffers: shared hit=10
-> Nested Loop (cost=1.00..9.05 rows=1 width=8) (actual time=0.042..0.042 rows=1 loops=1)
Buffers: shared hit=10
-> Index Scan using index_on_routes_lower_path on public.routes routes_3 (cost=0.56..4.58 rows=1 width=4) (actual time=0.024..0.024 rows=1 loops=1)
Index Cond: (lower((routes_3.path)::text) = 'gitlab-org/gitlab'::text)
Filter: ((routes_3.source_type)::text = 'Project'::text)
Rows Removed by Filter: 0
Buffers: shared hit=5
-> Index Only Scan using projects_pkey on public.projects projects_3 (cost=0.43..4.45 rows=1 width=4) (actual time=0.010..0.010 rows=1 loops=1)
Index Cond: (projects_3.id = routes_3.source_id)
Heap Fetches: 1
Buffers: shared hit=5
-> Index Scan using index_cross_projects_artifacts on public.ci_builds ci_builds_7 (cost=0.70..2.88 rows=1 width=8) (actual time=2.219..2.219 rows=0 loops=1)
Index Cond: ((ci_builds_7.project_id = projects_3.id) AND ((ci_builds_7.name)::text = 'coverage'::text) AND ((ci_builds_7.ref)::text = '36776-entropy-requirements-for-new-user-passwords-mvc'::text))
Buffers: shared hit=3 read=2
I/O Timings: read=2.187
-> Index Scan using ci_builds_pkey on public.ci_builds ci_builds_6 (cost=0.57..4.59 rows=1 width=1504) (actual time=0.000..0.000 rows=0 loops=1)
Index Cond: (ci_builds_6.id = (max(ci_builds_7.id)))
Filter: ((ci_builds_6.type)::text = 'Ci::Build'::text)
Rows Removed by Filter: 0
-> Nested Loop (cost=12.52..16.57 rows=1 width=1504) (actual time=5.446..5.446 rows=0 loops=1)
Buffers: shared hit=12 read=3
I/O Timings: read=5.352
-> Aggregate (cost=11.95..11.96 rows=1 width=4) (actual time=5.439..5.439 rows=1 loops=1)
Buffers: shared hit=12 read=3
I/O Timings: read=5.352
-> Nested Loop (cost=9.74..11.95 rows=1 width=4) (actual time=5.436..5.436 rows=0 loops=1)
Buffers: shared hit=12 read=3
I/O Timings: read=5.352
-> HashAggregate (cost=9.05..9.06 rows=1 width=8) (actual time=0.038..0.039 rows=1 loops=1)
Group Key: projects_4.id
Buffers: shared hit=10
-> Nested Loop (cost=1.00..9.05 rows=1 width=8) (actual time=0.032..0.033 rows=1 loops=1)
Buffers: shared hit=10
-> Index Scan using index_on_routes_lower_path on public.routes routes_4 (cost=0.56..4.58 rows=1 width=4) (actual time=0.018..0.019 rows=1 loops=1)
Index Cond: (lower((routes_4.path)::text) = 'gitlab-org/gitlab'::text)
Filter: ((routes_4.source_type)::text = 'Project'::text)
Rows Removed by Filter: 0
Buffers: shared hit=5
-> Index Only Scan using projects_pkey on public.projects projects_4 (cost=0.43..4.45 rows=1 width=4) (actual time=0.006..0.006 rows=1 loops=1)
Index Cond: (projects_4.id = routes_4.source_id)
Heap Fetches: 1
Buffers: shared hit=5
-> Index Scan using index_cross_projects_artifacts on public.ci_builds ci_builds_9 (cost=0.70..2.88 rows=1 width=8) (actual time=5.394..5.394 rows=0 loops=1)
Index Cond: ((ci_builds_9.project_id = projects_4.id) AND ((ci_builds_9.name)::text = 'dependency_scanning'::text) AND ((ci_builds_9.ref)::text = '31034-cache-classes-in-ci'::text))
Buffers: shared hit=2 read=3
I/O Timings: read=5.352
-> Index Scan using ci_builds_pkey on public.ci_builds ci_builds_8 (cost=0.57..4.59 rows=1 width=1504) (actual time=0.001..0.001 rows=0 loops=1)
Index Cond: (ci_builds_8.id = (max(ci_builds_9.id)))
Filter: ((ci_builds_8.type)::text = 'Ci::Build'::text)
Rows Removed by Filter: 0
Time: 113.609 ms
- planning: 58.040 ms
- execution: 55.569 ms
- I/O read: 53.708 ms
- I/O write: 0.000 ms
Shared buffers:
- hits: 50 (~400.00 KiB) from the buffer pool
- reads: 27 (~216.00 KiB) from the OS file cache, including disk I/O
- dirtied: 3 (~24.00 KiB)
- writes: 0Full execution plan for first query fragment
Nested Loop (cost=12.52..16.57 rows=1 width=1504) (actual time=0.124..0.124 rows=0 loops=1)
Buffers: shared hit=15
-> Aggregate (cost=11.95..11.96 rows=1 width=4) (actual time=0.123..0.123 rows=1 loops=1)
Buffers: shared hit=15
-> Nested Loop (cost=9.74..11.95 rows=1 width=4) (actual time=0.122..0.122 rows=0 loops=1)
Buffers: shared hit=15
-> HashAggregate (cost=9.05..9.06 rows=1 width=8) (actual time=0.091..0.091 rows=1 loops=1)
Group Key: projects.id
Buffers: shared hit=10
-> Nested Loop (cost=1.00..9.05 rows=1 width=8) (actual time=0.086..0.087 rows=1 loops=1)
Buffers: shared hit=10
-> Index Scan using index_on_routes_lower_path on public.routes (cost=0.56..4.58 rows=1 width=4) (actual time=0.037..0.037 rows=1 loops=1)
Index Cond: (lower((routes.path)::text) = 'gitlab-org/gitlab'::text)
Filter: ((routes.source_type)::text = 'Project'::text)
Rows Removed by Filter: 0
Buffers: shared hit=5
-> Index Only Scan using projects_pkey on public.projects (cost=0.43..4.45 rows=1 width=4) (actual time=0.044..0.045 rows=1 loops=1)
Index Cond: (projects.id = routes.source_id)
Heap Fetches: 1
Buffers: shared hit=5
-> Index Scan using index_cross_projects_artifacts on public.ci_builds ci_builds_1 (cost=0.70..2.88 rows=1 width=8) (actual time=0.030..0.030 rows=0 loops=1)
Index Cond: ((ci_builds_1.project_id = projects.id) AND ((ci_builds_1.name)::text = 'code_quality'::text) AND ((ci_builds_1.ref)::text = 'cross-projects-artifacts-dependencies'::text))
Buffers: shared hit=5
-> Index Scan using ci_builds_pkey on public.ci_builds (cost=0.57..4.59 rows=1 width=1504) (actual time=0.000..0.000 rows=0 loops=1)
Index Cond: (ci_builds.id = (max(ci_builds_1.id)))
Filter: ((ci_builds.type)::text = 'Ci::Build'::text)
Rows Removed by Filter: 0Time: 19.492 ms
- planning: 19.153 ms
- execution: 0.339 ms
- I/O read: 0.000 ms
- I/O write: 0.000 ms
Shared buffers:
- hits: 15 (~120.00 KiB) from the buffer pool
- reads: 0 from the OS file cache, including disk I/O
- dirtied: 0
- writes: 0Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. - [-] Tested in all supported browsers
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
- [-] Label as security and @ mention
@gitlab-com/gl-security/appsec - [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
- [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Marcel Amirault