Introduce Git only crawling to CMBR for HTTPs traffic
Adds traffic option to CMBR with web value by default. When specified as git it will only send git related HTTP traffic. It goes through projects via API and sends requests to git-upload-pack and git-receive-pack services. These requests need basic auth token to authorise.
Example output with debug:
go run cmbr.go --instance https://staging.gitlab.com/api/v4/projects --concurrency 1 --duration 1m -output color -git -debug
DEBU[0000] Creating request for https://staging.gitlab.com/api/v4/projects url="https://staging.gitlab.com/api/v4/projects"
DEBU[0000] Adding link url="https://staging.gitlab.com/gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-7e930446c00f4710.git/info/refs?service=git-upload-pack"
DEBU[0000] Adding link url="https://staging.gitlab.com/gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-7e930446c00f4710.wiki.git/info/refs?service=git-upload-pack"
DEBU[0000] Adding link url="https://staging.gitlab.com/gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-7e930446c00f4710.git/info/refs?service=git-receive-pack&repository_path=gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-7e930446c00f4710"
DEBU[0000] Adding link url="https://staging.gitlab.com/api/v4/projects/7675467"
DEBU[0000] Creating request for https://staging.gitlab.com/gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-7e930446c00f4710.git/info/refs?service=git-upload-pack url="https://staging.gitlab.com/gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-7e930446c00f4710.git/info/refs?service=git-upload-pack"
DEBU[0000] Adding link url="https://staging.gitlab.com/gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-15b507ab4d53572c.git/info/refs?service=git-upload-pack"
Worth calling out that some of these requests may fail when target project doesn't have wiki or overall the project was deleted.
Verification that git traffic is now generated
Closes #4 (closed)
Edited by Nailia Iskhakova
