## Abstract Unbounded data growth threatens GitLab.com's stability, inflates infrastructure costs, and creates operational drag. Data Growth Controls give us a systematic way to manage the database as a finite resource rather than an infinite landfill. Large self-managed customers face the same challenges. This Epic is the single entry point for all PostgreSQL Data Growth Controls efforts. ## Timeline - `2026-07-31` - [Q2 Epic 2: Define retention policies for 50% of "over_limit" tables](https://gitlab.com/groups/gitlab-operating-model/-/work_items/895) ```mermaid gantt title This is an example wip dateFormat YYYY-MM-DD section Strategy section Policy Data Lifecycle Workflow :2026-06-16, 2026-07-01 SSOT documentation framework :2026-06-08, 2026-07-01 CI validation for schema changes :2026-07-01, 2026-07-15 Define retention policie :after Policy1, 90d section Implementation Bulk data removal tooling :2026-08-01, 2026-10-31 pg_repack automation :2026-10-31, 2027-01-31 Growth monitoring dashboards :2026-10-31, 2027-01-31 Data owner onboarding :2026-10-31, 2027-01-31 section Targets "Q2 Epic 2 - Define retention policies for 50% of over_limit tables" :after Policy1, 2026-07-31, 1d Epic due date :crit, 2026-10-31, 1d ``` <!-- - Q1 FY27: February 1, 2026 – April 30, 2026 (Earnings reported in June 2026). - Q2 FY27: May 1, 2026 – July 31, 2026. - Q3 FY27: August 1, 2026 – October 31, 2026. - Q4 FY27: November 1, 2026 – January 31, 2027 --> ## Goals The goal of PostgreSQL Data Growth Controls is to transition GitLab's databas usage from unbounded, ad-hoc data accumulation to a managed data lifecycle — where every table has an explicit retention policy, tooling exists to enforce it, and growth is a deliberate decision rather than an unmanaged side effect. Specifically, we aim to: 1. Prevent unbounded database growth — GitLab's PostgreSQL databases operate under a hard disk ceiling. Without proactive controls, individual tables grow unchecked until they become operational risks. Data growth controls ensure we stay within capacity limits with predictable runway. 2. Define retention policies for every table — Establish a centralized single source of truth (SSOT) where every table has a documented retention strategy: how long data lives, what triggers its removal, and who owns that decision. 3. Enforce retention at the point of creation — Shift left by requiring data retention specifications in schema changes (new tables and columns), so growth expectations are explicit at design time rather than discovered during triage. 4. Build tooling for bulk data removal — Provide frameworks and tools that make it safe, efficient, and routine to remove obsolete data at scale — including support for offloading to object storage and automated space reclamation via pg_repack. 5. Improve system performance and reliability — Smaller tables mean faster queries, smaller indexes, quicker migrations, and more predictable system behavior. Controlling data growth directly improves GitLab.com availability. 6. Reduce infrastructure costs — Less stored data means lower storage bills, cheaper backups, and reduced replication overhead. Data retention is a compounding cost lever. 7. Enable compliance and auditability — Explicit retention policies provide the foundation for meeting regulatory requirements around data handling and give clear answers about what data is kept, for how long, and why. 8. Enable other teams without becoming the bottleneck — The Database Architecture team builds the framework, tools, and examples; data owners across GitLab adopt and enforce retention policies for their own domains. ## Structure - [PostgreSQL Data Growth Controls (this)](https://gitlab.com/groups/gitlab-org/-/work_items/22105) - [PostgreSQL Data Growth Controls - Strategy](...) - Team relationship: How me manage collaboration with other teams - Product management - SaaS - Self-managed - [PostgreSQL Data Growth Controls - Policy](...) - A Unified data lifecycle management workflow will be devised as part of the implementation. - Define retention policies for ("over_limit") tables - [PostgreSQL Data Growth Controls - Implementation](https://gitlab.com/groups/gitlab-org/-/work_items/21065) * [Deliverables](https://gitlab.com/groups/gitlab-org/-/work_items/21065#deliverables) (milestone wise) ## Scalability Forecasts This epic is public, Scalability Forecasts are tracked in this internal note [Scalability Forecasts (internal)](https://gitlab.com/groups/gitlab-org/-/work_items/22105#note_3394789236). ## Exit Criteria Data Growth Controls can be considered complete when: - [ ] 100% of tables have a documented retention policy — Every table has an explicit retention classification in the database docs SSOT (e.g., time_decay, user_lifecycle, transient, indefinite_with_justification). - [ ] Retention is enforced at schema change time — Migrations that add new tables or columns cannot merge without a retention specification. CI validation rejects schema changes that lack a retention policy. - [ ] No tables exceed the 100 GB on-disk target without an approved exception — Tables currently over the limit have either been reduced through retention enforcement, partitioning, or offloading, or have a documented justification and a plan. - [ ] Bulk data removal tooling is production-ready and in use — Frameworks for safe, throttled deletion of obsolete data are available, documented, and actively used. - [ ] Automated identification of tables requiring retention policies — Tooling continuously monitors table growth and flags tables that are approaching limits or lack retention policies, without manual analysis. - [ ] Space reclamation is automated — pg_repack (or equivalent) runs on a regular schedule to reclaim disk space after retention-driven deletions, without manual intervention. - [ ] Data owners manage their own retention — Feature teams define, implement, and maintain retention policies for their domains using the framework and tooling provided. The Database Architecture team is consulted, not required, for routine retention work. - [ ] Impact is measurable and measured — Dashboards track total database size, per-table growth rates, retention policy coverage, and infrastructure cost impact. Leadership can see progress without asking.