Reduce process-exporter scrape interval on Redis nodes
gitlab-org/gitlab#360578 (comment 966091343) says:
I was able to catch one of these events while running pidstat at 1 second granularity:
Currently we're using process-exporter with the default scrape interval (1 minute) as far as I can tell: https://prometheus.gprd.gitlab.net/targets#pool-process_exporter
This means that our process-exporter metrics cannot catch a spike like this as they are not granular enough.
We could try reducing the scrape interval for process-exporter on the Redis nodes (or even just the cache ones) to see if that changes our view of the CPU saturation. We don't yet know what resolution we'd need to catch this behaviour.
https://gitlab.com/gitlab-com/gl-infra/chef-repo/-/blob/da71bb246f63e06c683fa0757454fab10b785b4c/roles/gprd-infra-prometheus-server.json#L179-183 is where we set this globally but we do not want to change this globally as everything runs process-exporter, and that's too risky a change.