As raised in the Engineering Allocation call (reference https://docs.google.com/document/d/1j_9P8QlvaFO-XFoZTKZQsLUpm1wA2Vyf_Y83-9lX9tg/edit#bookmark=id.1rw0kigyk61u). The Tamland report is showing that one of the fastest growing tables on GitLab.com is `ci_builds_metadata`.  https://gitlab-com.gitlab.io/gl-infra/tamland/utilization.html#patroni-service-pg_table_size-bytes Growth is fairly linear. Tamland is forecasting ~44% growth in the next 60 days. ## Ask This growth rate is very high on an already very big table. We should investigate whether it's possible to optimize this, and how long this current growth rate is sustainable. ## Details One thing that is very unusual here is that the table growth is almost exactly the same month-on-month, at 9947 Gibibyte growth. This identical growth rate is strange. ~~Either the metric is broken, or some very unusual process is taking place in Postgres.~~ The fact that it is so linear is likely a clue as to the cause of the issue.  [source](https://thanos.gitlab.net/graph?g0.expr=increase(gitlab_component_utilization%3Apg_table_size_bytes%3Atopk%3A1h%7Bcomponent%3D%22pg_table_size%22%2Cenv%3D%22gprd%22%2Cenvironment%3D%22gprd%22%2Cmetrics_subsystem%3D%22utilization%22%2Cmonitor%3D%22db%22%2Cprovider%3D%22gcp%22%2Cregion%3D%22us-east%22%2Crelname%3D%22ci_builds_metadata%22%2Cresource_labels%3D%22relname%22%2Cstage%3D%22main%22%2CsummaryType%3D%22topk%22%2Ctier%3D%22db%22%2Ctype%3D%22patroni%22%2Cunit%3D%22bytes%22%7D%5B30d%5D)%20%2F%202%5E30&g0.tab=0&g0.stacked=0&g0.range_input=6w&g0.max_source_resolution=auto&g0.deduplicate=1&g0.partial_response=0&g0.store_matches=%5B%5D&g0.step_input=604800) ----------------------------------------------  Raw data: [source](https://thanos-query.ops.gitlab.net/graph?g0.expr=increase(pg_total_relation_size_bytes%7Benv%3D%22gprd%22%2C%20relname%3D%22ci_builds_metadata%22%7D%5B30d%5D)%20%2F%202%5E30%0A&g0.tab=0&g0.stacked=0&g0.range_input=8w&g0.max_source_resolution=0s&g0.deduplicate=1&g0.partial_response=0&g0.store_matches=%5B%5D) ### Technical Proposal / next steps 1. gitlab#351652 - This issue is part of the parent epic's work and a prerequisite to this work. 1. https://gitlab.com/groups/gitlab-org/-/epics/5417+ - In progress, see the epic for details. 1. Work from the [technical proposal](https://docs.google.com/document/d/1ARdoTZDy4qLGf6Z1GIHh83-stG_ZLpqsibjKr_OXMgc/edit) (this epic) - More to come after step 2 is completed. cc @alexives @sgoldstein @craig-gomes @abrandl