Database Group - 14.6 Planning
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.
Capacity
There is no known noteworthy PTO that may impact the capacity for the planned milestone
Boards
Planning
Our top priority for %14.6 is to finish anything remaining on supporting multiple databases. We plan to start working on the batched background migration general availability and continue our work on automated database testing with production clones by adding support for (sidekiq) background migration testing.
%14.6
Top Priorities for Update all our tools to work with multiple databases
DRI for all topics under this initiative: @pbair
-
Decomposition: Update background migrations to work with multiple databases
Who:
@pbair
@abrandl
-
Decomposition: Update database tooling to support multiple databases
Who: all Database Group members - priority pick for
@dfrazao-gitlab
@krasio
@stomlinson
as long as there are pending issues
Batched Background Migrations - General Availability
Who: @dfrazao-gitlab
@krasio
Priority topics for %14.6:
- Make Batched Background Migrations work with multiple databases
- Stability improvements
Automated database testing: Advanced database testing features (version 3)
Who: @stomlinson
@dfrazao-gitlab
Current open issues:
- Record transaction durations during migrations (gitlab-com-database-testing#33 (closed))
- Capture migration output as an artifact (gitlab-com-database-testing#18)
Priority topics for %14.6:
- Report statistics for background migration sidekiq jobs (gitlab-com-database-testing#30 (closed))
- Record and analyze the locks taken during a migration (gitlab-com-database-testing#38)
Additional topics to be considered outside our long term initiatives
- Review moving the
GITLAB_GRAFANA_API_KEY
environment variable into gitlab.rb configuration (gitlab-org/gitlab#332696 (closed))
%14.6
- Temporarily Deprioritized && Stretch goals
Secondary Priorities for Reduce tables sizes to < 100 GB per physical table
- Who:
@abrandl
Our up and coming top priority. Based on our analysis, we are planning to start working with multiple other GitLab teams in an ongoing fashion towards achieving that goal.
We may have to wait for Batched Background Migrations - General Availability, so that we can rebuild our partitioning helpers and other tools required.
To get this effort started, we plan to consolidate our metrics, so that we can have a clear view on the tables that may cause performance issues:
Database Schema Validation - ensure the deployed database schema matches codebase expectations
- Who:
@abrandl
- Next step: Implement index schema validation
For consideration
Deprecate custom name for default PG schema
We are not explicit about the default schema used by GitLab. We assume it to be public
, but it can also be set by setting the default path for the gitlab
user to a different schema.
This has caused issues in the past on self managed instances that ended up with multiple schemas used (due to accidental changes to the gitlab
user or other types of mismanagement)
We also want to be able to introduce multiple different PG schemas in the future, so being explicit about schema names makes everything less complicated and simpler to reason about.