Formalize Secret Push Protection performance tests into GPT

Overview

While building the Secret Push Protection feature, we had to manually conduct performance tests using GitLab Performance Tool (GPT) on a number of occasions (1, 2) to make sure the feature is performant and won't cause incidents or regressions as it's part of the "hot path" for git push operations in GitLab (whether via http/s or ssh).

However, since the feature has been released in GA, we want to integrate those performance tests into GPT to ensure they run in the same candence as other requests, and be part of the automated benchmarks for all of our reference architectures.

This will be very helpful for keeping track of the performance of the feature as we continue to polish and enhance the user experience, and for discussions with customers wishing to enable the feature on their instances but are worried it may degrade their developers' experience.

You can read more on GitLab's git push/pull request cycle and check this runbook for an idea on how Secret Push Protection fits into that.

Proposal

To run the performance tests, we had to create data and a separate test under the pre-receive folder in secret-detection branch of GPT repository. In order to make this test part of the automated benchmarks, we will likely need to:

• Submit a merge request to get the test to be part of the GPT repository:
  • Update the test code to account for any changes in the feature since the test was written.
  • Update the [test code] as discussed with @niskhakova in this comment.
  • Ensure the test is being run automatically as other tests.
  • Ensure the test output is included in the reference architectures' benchmarks.
  • Ensure the test output is included in the Gitlab versions' benchmarks
• Update Secret Push Protection – Performance Testing runbook to reflect this had been automated.