Partitioning: Initiate the project_daily_statistics partitioning
Background
Related to: Source Code: Table Cleanup: Partition project_d... (&18879 - closed)
The project_daily_statistics table stores daily aggregated metrics for GitLab projects. Due to the rapidly increasing table size, we decided to partition this table to improve performance and manageability.
This step creates the partitioned table infrastructure, sets up sync triggers, and begins the data migration process.
To-Do
- Create a new partitioned table
project_daily_statistics_part_[random_id]with nearly identical schema partitioned by date column - Create initial partitions covering the last 30 days of data (optimization to reduce migration time)
- Install sync triggers to mirror writes from original table to partitioned table
- Schedule batched background migration to copy historical data from the last 30 days only
resources:
Edited by Emma Park