Show the metric "The projects count per platform target" on Periscope dashboard - snowplow
Proposal
We want to get the results of a query like this
SELECT
ci_variables.value
, COUNT(*)
FROM ci_variables
WHERE ci_variables.key = 'AUTO_DEVOPS_PLATFORM_TARGET'
GROUP BY ci_variables.value;
And get that data into periscope. This can be done by stuffing that data into a new table that can then be exposed to periscope.
Merge Requests
- MR to add the new table.
- MR to add a background task that will update that table on a schedule, perhaps once a day.
Note: Follow-up work after this is merged to get data into periscope was detailed in this comment: !40036 (comment 401867984)
Original Proposal
Problem to solve
As a PM, I want to know that what deployment targets users are using in the AutoDevOps flow.
Intended users
User experience goal
Proposal
- We add a new column
cloud_service_type
(enum/integer) todeployments
table. It takes one ofeks, gke, ecs, fargate
. We add more types in the future e.g.ec2
, . - We extend
environment
keyword to specify a cloud service type for each deployment job.- We add
environment:cloud_service:
keyword, which persists the value intodeployments.cloud_service_type
.
- We add
- We expose
CI_CLUSTER_CLOUD_SERVICE_TYPE
predefined variable fromClusters::Cluster
model, which returns the provider type of GitLab Kubernetes Cluster Integration (i.e.deployment_platform.cluster.provider_type
) - Supports variable expansion in
environment: cloud_service
keywords.
Then, we extend the AutoDevOps templates for tracking the deployment targets:
diff --git a/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml b/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml
index cff4a15e70e..cce76a11d72 100644
--- a/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml
+++ b/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml
@@ -1,6 +1,8 @@
.auto-deploy:
image: "registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image:v0.17.0"
dependencies: []
+ environment:
+ cloud_service: $CI_CLUSTER_CLOUD_SERVICE_TYPE
include:
- template: Jobs/Deploy/ECS.gitlab-ci.yml
diff --git a/lib/gitlab/ci/templates/Jobs/Deploy/ECS.gitlab-ci.yml b/lib/gitlab/ci/templates/Jobs/Deploy/ECS.gitlab-ci.yml
index da474f8ac88..d603768e610 100644
--- a/lib/gitlab/ci/templates/Jobs/Deploy/ECS.gitlab-ci.yml
+++ b/lib/gitlab/ci/templates/Jobs/Deploy/ECS.gitlab-ci.yml
@@ -18,6 +18,7 @@
extends: .deploy_to_ecs
environment:
name: review/$CI_COMMIT_REF_NAME
+ cloud_service: ecs
.production_ecs_base:
stage: production
@@ -40,6 +41,8 @@ review_ecs:
review_fargate:
extends: .review_ecs_base
+ environment:
+ cloud_service: fargate
rules:
- if: '$AUTO_DEVOPS_PLATFORM_TARGET != "FARGATE"'
when: never
@@ -64,6 +67,8 @@ production_ecs:
production_fargate:
extends: .production_ecs_base
+ environment:
+ cloud_service: fargate
rules:
- if: '$AUTO_DEVOPS_PLATFORM_TARGET != "FARGATE"'
when: never
This effectively allows the following queries to track the deployment usage:
-
Deployment.success.count
... Total successful deployments -
Deployment.cloud_service_type_eks.success.count
... Total successful deployments to EKS on AWS -
Deployment.cloud_service_type_gke.success.count
... Total successful deployments to GKE on GCP -
Deployment.cloud_service_type_ecs.success.count
... Total successful deployments to ECS on AWS -
Deployment.cloud_service_type_fargate.success.count
... Total successful deployments to Fargate on AWS
NOTE: You likely need to create a database index on cloud_service_type
column for optimizing the above query, otherwise the query could get aborted by statement timeout.
Further details
This annotation to the deployment jobs allow us to implement advanced feature for automating/smoothing the bootstrap for deployments to multiple-cloud services.
Following https://gitlab.com/gitlab-org/telemetry/-/issues/382#note_342150277 we need to add a DB entry to be able to identify which deployment platforms users are using.