<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> *This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.* <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> We are [shifting left](https://about.gitlab.com/topics/ci-cd/shift-left-devops/) our ability to pre-emptively find database related regressions and performance issues by testing all database updates against a production clone of GitLab.com's database. With every feature we add, we move one step closer to GitLab being more performant and lower the risk that code may be deployed that could cause incidents and affect the performance and availability of GitLab.com or other self managed instances. Related Blueprint: https://docs.gitlab.com/ee/architecture/blueprints/database_testing ### Status Updates %"15.0" marked an important milestone for the Automated database testing GitLab internal feature - we are now testing **all types** of database migrations against a clone of the production database of GitLab.com :rocket: :tada: - All regular and post migrations - all schema updates and small scale data updates - All data migrations (through sampling) - both Sidekiq and Batched background migrations That means that 100% of _**scheduled**_ database updates are covered, making sure that we test our most tricky operations before they are even merged. The impact of this feature to the availability and reliability of GitLab.com and large GitLab instances has been so significant that can only be observed by the fact that we have kept major database related incidents to a minimum and we are allowing GitLab.com to keep on scaling at an always increasing rate without having to face major database outbreaks. The effect of those features should also be evident to self managed instances throughout GitLab 15 and beyond. But that was only the beginning as we are expanding our scope; _**scheduled**_ database updates do not include regular queries or updates that result from user interactions (users performing an action or causing a background job to run). Our next effort will be to find ways to perform [automated query analysis for Merge Requests](https://gitlab.com/groups/gitlab-org/database-team/-/epics/12) and test newly introduced queries against our production clones as well. This is a difficult problem to solve as we must figure out the parameters for the queries, which depend on the data stored, so we are going to start with the simplest iteration possible, identifying the queries introduced by each MR to support the database reviewers.