Introduce Git only crawling to CMBR for HTTPs traffic
Adds traffic
option to CMBR with web
value by default. When specified as git
it will only send git related HTTP traffic. It goes through projects via API and sends requests to git-upload-pack
and git-receive-pack
services. These requests need basic auth token to authorise.
Example output with debug:
go run cmbr.go --instance https://staging.gitlab.com/api/v4/projects --concurrency 1 --duration 1m -output color -git -debug
DEBU[0000] Creating request for https://staging.gitlab.com/api/v4/projects url="https://staging.gitlab.com/api/v4/projects"
DEBU[0000] Adding link url="https://staging.gitlab.com/gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-7e930446c00f4710.git/info/refs?service=git-upload-pack"
DEBU[0000] Adding link url="https://staging.gitlab.com/gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-7e930446c00f4710.wiki.git/info/refs?service=git-upload-pack"
DEBU[0000] Adding link url="https://staging.gitlab.com/gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-7e930446c00f4710.git/info/refs?service=git-receive-pack&repository_path=gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-7e930446c00f4710"
DEBU[0000] Adding link url="https://staging.gitlab.com/api/v4/projects/7675467"
DEBU[0000] Creating request for https://staging.gitlab.com/gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-7e930446c00f4710.git/info/refs?service=git-upload-pack url="https://staging.gitlab.com/gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-7e930446c00f4710.git/info/refs?service=git-upload-pack"
DEBU[0000] Adding link url="https://staging.gitlab.com/gitlab-qa-sandbox-group-3/qa-test-2022-06-14-11-18-24-72872b8f413209f7/push_rules-15b507ab4d53572c.git/info/refs?service=git-upload-pack"
Worth calling out that some of these requests may fail when target project doesn't have wiki or overall the project was deleted.
Verification that git traffic is now generated
Closes #4 (closed)
Edited by Nailia Iskhakova