Skip to content

Introduce Git only crawling to CMBR for HTTPs traffic

Based on analysis in gitlab-org/gitlab#351229 (closed), we need to increase Git-specific load. The issue is to explore limiting crawling to Git only requests similarly to -api option.

Steps

  • Add git only option that will run Git related requests
  • Collect projects urls and add Git specific endpoints to it
  • Add Git load job similar to generate_load_api or generate_load_web, (target RPS = 70 based on this analysis)

Gitaly endpoints

Git specific GET request endpoints:

  • project url + .git/info/refs?service=git-upload-pack (example - https://staging.gitlab.com/gpt/large_projects/gitlabhq1.git/info/refs?service=git-upload-pack)
  • project url + .wiki.git/info/refs?service=git-upload-pack (example - https://staging.gitlab.com/gpt/large_projects/gitlabhq1.wiki.git/info/refs?service=git-upload-pack)

More involved GET endpoint that require basic auth header (Authorization: Basic <code>'):

Where <code> is the result of echo -n "user:password" | base64

Edited by Nailia Iskhakova