Consider capacity planning for runway prometheus stateful sets
The related incident showed a marked increase in memory usage, possibly from an increase in the volume of metrics being collected and served. We should consider a system to scale the resources for these pods to avoid a critical failure.
_This ticket was created from_ [_INC-6781_](https://app.incident.io/gitlab/incidents/6781) _using_ [_incident.io_](https://app.incident.io) 🔥
issue