Change the kubernetes_agents_with_tokens metric to represent only active tokens
Summary
The current representation of kubernetes_agents_with_tokens
is not currently relevant to our product. We want to have the count of active agents instead.
Proposal
Here's an example query representation of what we want to achieve:
gitlabhq_dblab=# SELECT MIN("cluster_agent_tokens"."agent_id") FROM "cluster_agent_tokens" WHERE "cluster_agent_tokens"."status" = 0 AND "cluster_agent_tokens"."last_used_at" > '2022-11-01 11:28:54.819262+00';
min|79
gitlabhq_dblab=# SELECT MAX("cluster_agent_tokens"."agent_id") FROM "cluster_agent_tokens" WHERE "cluster_agent_tokens"."status" = 0 AND "cluster_agent_tokens"."last_used_at" > '2022-11-01 11:28:54.819262+00';
max|38392
gitlabhq_dblab=# SELECT COUNT(DISTINCT "cluster_agent_tokens"."agent_id") FROM "cluster_agent_tokens" WHERE "cluster_agent_tokens"."status" = 0 AND "cluster_agent_tokens"."last_used_at" > '2022-11-01 11:28:54.819262+00' AND "cluster_agent_tokens"."agent_id" >= 79 AND "cluster_agent_tokens"."agent_id" < 38392;
count|6737
We should change this current query, to represent the proposal above. That is, adding to the query:
"cluster_agent_tokens"."status" = 0
"cluster_agent_tokens"."last_used_at" > NOW() - interval '24 hours'
Note that NOW() - interval '24 hours'
should be calculated in Rails before passing to the query, not in database, as this affect the query drastically. Here's the query explain with the date pre-calculated.
References
- This grafana graph shows the correct number on GitLab SaaS
/cc @nagyv-gitlab @nmezzopera for milestone planning
Edited by Viktor Nagy (GitLab)