Support custom cookies in crawler

Support setting custom cookies to requests. Specifically it can help to target Staging Canary if needed.

Also update docs to group al env variables in a list.

Related to gitlab-org/gitlab#338978 (closed)