Fix environment status in merge request widget
What does this MR do?
This MR is to adjust the current implementation between https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22292 and https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22380 in order to accomplish https://gitlab.com/gitlab-org/gitlab-ce/issues/25140 Deliverable
This MR is a follow-up from https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22380.
ci_environments_status
to get the real latest deployment
Fix When ci_environments_status
is accessed, it returns the latest deployment through existing refs (refs/environments/${env_name}/deployments/${deployment_iid}
). This means we cannot get running
/failed
/etc deployments, but only success
deployments, because the ref is created when the deployment succeeded.
To get the real latest deployment record (and its status), we have to ask database, instead of gitaly.
Proxy specific deployment statuses
Deployment
object has multiple statuses (created
/running
/success
/failed
/canceled
), however, frontend code can recognize running
/success
/failed
only. Because of that, deployment status in MR widget will be horrible appearance (See https://gitlab.com/gitlab-org/gitlab-ce/issues/53578#further-details) if it's either created
or canceled
.
Given frontend doesn't allocate capacity in 11.5, we decided to proxy unsupported statuses to supported ones in backend. Specifically, created
will be running
, canceled
will be failed
, respectively.
This is a temporary fix and will be followed-up in https://gitlab.com/gitlab-org/gitlab-ce/issues/53578.
DeadlineExceeded
(See more https://gitlab.com/gitlab-org/gitlab-ce/issues/51120)
Mitigate the error rates of Gitaly Recently, production engineers/SRE were suffering because the error rates of ci_environments_status
endpoint (gitlab-com/gl-infra/production#542 (closed)) was high (about 3000-5000 crash reports/day). The is because ci_environments_status
uses git reference to fetch the corresponding deployment record, however, as I explained above, this can be done in DB query. We can expect that we can mitigate the error rates.
Fix users cannot stop their environments
We accidentally removed BuildSuccessWorker
invocation in the downstream MR https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22380. We have to execute BuildSuccessWorker
always, otherwise, users cannot stop environments.
Query optimization
Here are the new queries introduced in this fix.
1. environment.deployments.start.find_by_sha(sha)
SELECT "deployments".* FROM "deployments" WHERE "deployments"."environment_id" = 34 AND "deployments"."sha" = '0c2ea7d19b24e2a1eb24d9165c0da0fd1cb25595' LIMIT 1;
2. Environment.where(project_id: [mr.source_project_id, mr.target_project_id]).available.with_deployment(mr.diff_head_sha)
SELECT "environments".* FROM "environments" WHERE "environments"."project_id" IN (15, 15) AND ("environments"."state" IN ('available')) AND (EXISTS (SELECT 1 FROM "deployments" WHERE (deployments.environment_id = environments.id) AND "deployments"."sha" = '0c2ea7d19b24e2a1eb24d9165c0da0fd1cb25595'));
Those queries can be optimized with the following index.
# Index
"index_deployments_on_environment_id_and_sha" btree (environment_id, sha)
# Query plan 1
gitlabhq_development_ce=# explain analyze SELECT "deployments".* FROM "deployments" WHERE "deployments"."environment_id" = 34 AND "deployments"."sha" = '0c2ea7d19b24e2a1eb24d9165c0da0fd1cb25595' LIMIT 1;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=0.28..8.30 rows=1 width=126) (actual time=0.015..0.015 rows=1 loops=1)
-> Index Scan using index_deployments_on_environment_id_and_sha on deployments (cost=0.28..8.30 rows=1 width=126) (actual time=0.015..0.015 rows=1 loops=1)
Index Cond: ((environment_id = 34) AND ((sha)::text = '0c2ea7d19b24e2a1eb24d9165c0da0fd1cb25595'::text))
Planning time: 0.121 ms
Execution time: 0.034 ms
(5 rows)
# Query plan 2
gitlabhq_development_ce=# explain analyze SELECT "environments".* FROM "environments" WHERE "environments"."project_id" IN (15, 15) AND ("environments"."state" IN ('available')) AND (EXISTS (SELECT 1 FROM "deployments" WHERE (deployments.environment_id = environments.id) AND "deployments"."sha" = '0c2ea7d19b24e2a1eb24d9165c0da0fd1cb25595'));
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Nested Loop Semi Join (cost=0.28..17.74 rows=1 width=105) (actual time=0.035..0.039 rows=1 loops=1)
-> Seq Scan on environments (cost=0.00..1.42 rows=1 width=105) (actual time=0.008..0.009 rows=7 loops=1)
Filter: ((project_id = ANY ('{15,15}'::integer[])) AND ((state)::text = 'available'::text))
Rows Removed by Filter: 30
-> Index Only Scan using index_deployments_on_environment_id_and_sha on deployments (cost=0.28..8.30 rows=1 width=4) (actual time=0.004..0.004 rows=0 loops=7)
Index Cond: ((environment_id = environments.id) AND (sha = '0c2ea7d19b24e2a1eb24d9165c0da0fd1cb25595'::text))
Heap Fetches: 1
Planning time: 0.393 ms
Execution time: 0.056 ms
(9 rows)
What are the relevant issue numbers?
Close https://gitlab.com/gitlab-org/gitlab-ce/issues/25140 Close https://gitlab.com/gitlab-org/gitlab-ce/issues/51120 Related https://gitlab.com/gitlab-org/gitlab-ce/issues/53578
This MR has EE port MR
Does this MR meet the acceptance criteria?
-
Changelog entry added, if necessary - [-] Documentation created/updated
-
Tests added for this feature/bug -
Conforms to the code review guidelines -
Conforms to the merge request performance guidelines -
Conforms to the style guides -
Conforms to the database guides - [-] Link to e2e tests MR added if this MR has Requires e2e tests label. See the Test Planning Process.