Skip to content

chore(metrics): Provision histogram-specific rollups via the Scheduler

Ankit Bhatnagar requested to merge abhatnagar/provision-histogram1m-rollup into main

Related https://gitlab.com/gitlab-org/opstrace/opstrace/-/issues/2574

This MR provisions aggregated rollups for histogram metrics via the Scheduler as warranted by the aforementioned issue. Tested on a dev-cluster with:

➜  abhtngr-dev git:(main) ✗ kubectl get cluster abhtngr-dev -o json | jq .status.conditions
[
  {
    "lastTransitionTime": "2024-03-19T10:37:43Z",
    "message": "All components are in ready state",
    "observedGeneration": 1,
    "reason": "Success",
    "status": "True",
    "type": "Ready"
  }
]
➜  abhtngr-dev git:(main) ✗ kubectl get cluster abhtngr-dev -o json | jq .status.lastMigrationApplied
"0.3.0-e381d43e"
cluster-0-1-0.cluster-0-1.default.svc.cluster.local :) show tables like '%histogram_1m%'

SHOW TABLES LIKE '%histogram_1m%'

Query id: c76d58cd-eb2f-414a-9e75-500526c24d64

┌─name─────────────────────────┐
│ metrics_main_histogram_1m    │
│ metrics_main_histogram_1m_mv │
└──────────────────────────────┘

2 rows in set. Elapsed: 0.240 sec.
cluster-0-1-0.cluster-0-1.default.svc.cluster.local :) select max(AggTimeUnix) from metrics_main_histogram_1m;

SELECT max(AggTimeUnix)
FROM metrics_main_histogram_1m

Query id: 6a9a2e28-19cc-4131-835b-88c9b4057099

┌──────────────max(AggTimeUnix)─┐
│ 2024-03-27 11:49:00.000000000 │
└───────────────────────────────┘

1 row in set. Elapsed: 0.245 sec. Processed 129.18 thousand rows, 1.03 MB (526.22 thousand rows/s., 4.21 MB/s.)

cluster-0-1-0.cluster-0-1.default.svc.cluster.local :) select max(IngestionTimestamp) from metrics_main_histogram;

SELECT max(IngestionTimestamp)
FROM metrics_main_histogram

Query id: 60cc2ac9-28c4-4a5b-b64c-ee88281c6a03

┌───────max(IngestionTimestamp)─┐
│ 2024-03-27 11:49:47.000117095 │
└───────────────────────────────┘

1 row in set. Elapsed: 0.248 sec. Processed 384.75 thousand rows, 3.08 MB (1.55 million rows/s., 12.42 MB/s.)
Edited by Ankit Bhatnagar

Merge request reports