Skip to content

feat: Add periodic query to track cloud spend

Bob Van Landuyt requested to merge bvl/add-cloud-spend-periodic-query into master

feat: Add periodic query to track cloud spend

This adds a periodic query that will upload cloud spend information to the regular bucket.

There it will be consumed into BigQuery to build a looker dashboard so the team can monitor this.

Resulting run:

(master → origin {28} U:2 ?:1 ✗) runbooks bundle exec scripts/perform-periodic-thanos-queries.rb 
Fetching results for cloud-cost-spend
-- ⬇ Performing query for cloud_cost_spend
Fetching results for daily-database-table-size
-- ⬇ Performing query for database_table_bloat_ratio_daily
-- ⬇ Performing query for database_table_size_daily
Fetching results for daily-haproxy-bytes
-- ⬇ Performing query for total_haproxy_bytes_out
Fetching results for daily-slas
-- ⬇ Performing query for overall_ratio
-- ⬇ Performing query for overall_target
-- ⬇ Performing query for service_ratio
Fetching results for error-budgets-performance-indicator
-- ⬇ Performing query for stage_group_error_budget_completeness
-- ⬇ Performing query for stage_group_error_budget_teams_over_budget_availability
Fetching results for stage-group-7d-error-budgets
-- ⬇ Performing query for stage_group_error_budget_availability
-- ⬇ Performing query for stage_group_error_budget_seconds_remaining
-- ⬇ Performing query for stage_group_error_budget_seconds_spent
Fetching results for stage-group-error-budgets
-- ⬇ Performing query for stage_group_error_budget_availability
-- ⬇ Performing query for stage_group_error_budget_seconds_remaining
-- ⬇ Performing query for stage_group_error_budget_seconds_spent
-- ⬇ Performing query for stage_group_traffic_share
Storing results for 7 topics in /home/bvl/repos/runbooks/periodic-query-results/20231215110238
-- 💾 cloud-cost-spend: /home/bvl/repos/runbooks/periodic-query-results/20231215110238/cloud-cost-spend.json
-- 💾 daily-database-table-size: /home/bvl/repos/runbooks/periodic-query-results/20231215110238/daily-database-table-size.json
-- 💾 daily-haproxy-bytes: /home/bvl/repos/runbooks/periodic-query-results/20231215110238/daily-haproxy-bytes.json
-- 💾 daily-slas: /home/bvl/repos/runbooks/periodic-query-results/20231215110238/daily-slas.json
-- 💾 error-budgets-performance-indicator: /home/bvl/repos/runbooks/periodic-query-results/20231215110238/error-budgets-performance-indicator.json
-- 💾 stage-group-7d-error-budgets: /home/bvl/repos/runbooks/periodic-query-results/20231215110238/stage-group-7d-error-budgets.json
-- 💾 stage-group-error-budgets: /home/bvl/repos/runbooks/periodic-query-results/20231215110238/stage-group-error-budgets.json
(master → origin {28} U:2 ?:1 ✗) runbooks cat periodic-query-results/20231215110238/cloud-cost-spend.json 
{"cloud_cost_spend":{"success":true,"status_code":"200","message":"OK","body":{"status":"success","data":{"resultType":"vector","result":[{"metric":{"feature_category":"code_suggestions","item":"completions/completion/input","unit":"characters","vendor":"anthropic"},"value":[1702638147,"6593456.048190043"]},{"metric":{"feature_category":"code_suggestions","item":"completions/completion/input","unit":"characters","vendor":"vertex-ai"},"value":[1702638147,"47663625.48696486"]},{"metric":{"feature_category":"code_suggestions","item":"completions/completion/output","unit":"characters","vendor":"anthropic"},"value":[1702638147,"174911.3526423506"]},{"metric":{"feature_category":"code_suggestions","item":"completions/completion/output","unit":"characters","vendor":"vertex-ai"},"value":[1702638147,"568831.6766386402"]}]}}}}
Edited by Bob Van Landuyt

Merge request reports