fix: Throttle Pages RPS to bypass rate limits
The MR adds delay for Pages requests to throttle RPS. With 1 s delay it's more easy to predict how concurrency will affect the RPS. Here's the example of 10 minutes run with concurrency 15 - no 429 errors
go run cmbr.go --instance https://pages_test.staging.gitlab.io -concurrency 15 -duration 10m -traffic pages
INFO[0011] crawling... count=100
INFO[0021] crawling... count=200
INFO[0029] crawling... count=300
INFO[0038] crawling... count=400
...
INFO[0550] crawling... count=5600
INFO[0558] crawling... count=5700
INFO[0566] crawling... count=5800
INFO[0575] crawling... count=5900
INFO[0583] crawling... count=6000
INFO[0591] crawling... count=6100
Also verified that throttle doesn't affect other traffic options.
Related Slack discussion: https://gitlab.slack.com/archives/C1BSEQ138/p1658243931251419?thread_ts=1658236359.724929&cid=C1BSEQ138
Related to #9