## Problem GitLab has multiple tools used for validating performance, but they are most often used in scheduled pipelines using the `nightly` build, and not as part of the usual development + release cycle to `.com`. This means that issues can be deployed to Production and cause Incidents before they have a chance to be caught in downstream pipelines. ## Proposal Apdex is measured against APIs. GitLab Performance Tool (GPT) is our primary means of testing the performance of those APIs, so to protect Apdex and customer experience across all GitLab platforms, we should make passing GPT tests a requirement of committing code to GitLab. ### Deliverables | Phase | Epic/Issue | DRI | Priority | Success | |-------|------------|-----|----------|---------| | Phase 1 - Make it Work | [Make GPT available in GDK via the `gdk measure` command or a new one](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/3709) | @bwilkerson13 | ~"priority::1" | GPT can be invoked from the command line using a gdk command. | | | [Establish a default test settings/profile in GPT for GDK](https://gitlab.com/groups/gitlab-org/quality/quality-engineering/-/epics/159) | @john.mcdonnell | ~"priority::1" | Create the required option + environment files for GPT to run against a GDK environment | | | [Selective test execution to lower GPT test run times from 90+ minutes](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/3716) | @vishal.s.patel | ~"priority::1" | GPT can run tests selectively, generate a selective test command from whatever code is pending to be committed. | | Phase 2 - Make it Blocking | Make successful test runs part of the pre-commit hooks for appropriate commits. | | ~"priority::1" | GPT test results block a non-performant commit | | Phase 3 - Make it Better | Increase GPT endpoint coverage | | ~"priority::2" | Expand GPT tests to cover more endpoints | | | Figure out a way to speed up data seeding process | | ~"priority::2" | Reduce the amount of time to seed data for GPT into a clean environment | ## Challenges - Different developer machine specs might affect results. Can we meaningfully control this variable? [Test consistency is critical](https://grafana.com/docs/k6/latest/testing-guides/automated-performance-testing/#test-consistency-is-critical) - GDK already has performance challenges in people's environments, will it be able to run any meaningful performance tests? - Performance testing is typically a fuzzy process, focused on trend analysis over time. Given the other constraints and challenges, can we make a pass/fail threshold that is reliable enough to be blocking, and sensitive enough to catch anything? ## Success Metrics - Increased use of GPT in local environment to proactively performance test changes. - Reduced number of Apdex alerts and Incidents. ## Additional Context ### What DevEx Initiatives does "Make passing GitLab Performance Tool results a pre-commit requirement in GDK" target? * Prevent bugs from reaching customers, SREs, and Release Managers by, * Implementing Performance analysis to enable teams to reduce the number of performance-related incidents or bugs being reported by customers ### How? The outlined approach transforms performance testing from a downstream validation step into an integral part of the development workflow, creating a feedback loop that improves both code quality and team performance awareness. With the use of **Pre-commit blocking mechanism** GPT tests become a requirement before code commits, allowing performance issues to be caught at the earliest possible stage - before they enter the codebase. This prevents: * Apdex degradation that impacts customer experience * Performance-related incidents that burden SREs and Release Managers * Deployment of problematic code to production We will **Standardize local testing** by establishing default test settings/profiles for GPT in GDK environments, providing consistent performance measurement across development teams. We will implement **Selective test execution** with targeted testing based on code changes for developers to get faster, relevant feedback about their specific changes rather than running full 90+ minute test suites. We will gain **Increased endpoint coverage** by enabling the creation of GPT tests to cover more endpoints, broadening performance visibility across the application. We will increase **Developer adoption** by making performance testing easily accessible through GDK to enable proactive performance analysis during active development. <!-- STATUS NOTE START --> ## Status 2025-06-25 This item will be going to the backlog per the team decision in https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/3738+. _Copied from https://gitlab.com/groups/gitlab-org/quality/quality-engineering/-/epics/150#note_2582756537_ <!-- STATUS NOTE END -->