Draft: Proposal to reduce and limit table sizes to < 100 GB for tables on GitLab.com
Why is this change being made?
This is a proposal to communicate and work on a target for maximum table sizes per physical table on GitLab.com.
This sets a clear, measurable goal and expectations. It needs to be re-evaluated continuously with GitLab.com growing. For details how this is being applied pragmatically, see the document.
We may also want to find practical measures to elevate this effort and get it prioritized. In similar context, we have already discussed to disallow adding new indexes to certain tables, for example. This can act as a means to drive attention to this problem and get a discussion going.
How will we organize work?
I have created gitlab-org&6211 for a proposal with a few examples:
- One epic per table violating the rule
- Document proposals and different solutions to reducing the table size in issues on those epics
(Nothing there yet - only a showcase how we can organize this type of work)
Who will drive this work?
I think groupdatabase should own the following:
- Work with development in to establish this target
- Monitor growth and situation on GitLab.com
- Find and propose solutions to reducing the table sizes and document those in detail in issues
- Work with the stage groups to prioritize and schedule those efforts
Links
Direct link to preview: https://ab-table-sizes.about.gitlab-review.app/handbook/engineering/development/enablement/database/doc/reduce-table-sizes.html