Formalize Secret Push Protection performance tests into GPT

Overview

While building the Secret Push Protection feature, we had to manually conduct performance tests using GitLab Performance Tool (GPT) on a number of occasions (1, 2) to make sure the feature is performant and won't cause incidents or regressions as it's part of the "hot path" for git push operations in GitLab (whether via http/s or ssh).

However, since the feature has been released in GA, we want to integrate those performance tests into GPT to ensure they run in the same candence as other requests, and be part of the automated benchmarks for all of our reference architectures.

This will be very helpful for keeping track of the performance of the feature as we continue to polish and enhance the user experience, and for discussions with customers wishing to enable the feature on their instances but are worried it may degrade their developers' experience.

You can read more on GitLab's git push/pull request cycle and check this runbook for an idea on how Secret Push Protection fits into that.

Proposal

To run the performance tests, we had to create data and a separate test under the pre-receive folder in secret-detection branch of GPT repository. In order to make this test part of the automated benchmarks, we will likely need to:

• Submit a merge request to get the test to be part of the GPT repository:
  • Update the test code to account for any changes in the feature since the test was written.
  • Rewrite the test to become a scenario where a new project is created on each test run instead of using existing projects from the test data. Similar to scenario_api_new_commits [context comment]
  • Create MR to enable Secret Detection on static performance environments - expand application configuration - add ApplicationSetting.last.update(pre_receive_secret_detection_enabled: true)
• Once new test GPT MR and Configuration MRs are merged
  • Ensure the test is being run automatically as other tests.
  • Ensure the test output is included in the reference architectures' benchmarks
• Update Secret Push Protection – Performance Testing runbook to reflect this had been automated.