Introduce floor threshold into our Capacity Planning process to improve financial efficiency

Claude: Capacity Planning is the strategic process of analyzing, forecasting, and provisioning IT resources to ensure systems meet performance requirements while optimizing costs through the elimination of both expensive overprovisioning and costly performance bottlenecks.

Our Capacity Planning process currently focusses on avoiding resource saturation, thus ensuring service availability. This is visible in Tamland by the use of our 'soft' and 'hard' SLO thresholds (example).

We should also consider introducing a 'low' threshold to help us understand when we might be over-provisioning resources, which can lead to cost wastage. This will help increase cost awareness across Engineering.

For cost reporting, we currently have access to Ternary (internal only) that helps visualize 'waste' for all of our GCP resources. This is a helpful tool, however sometimes resources show high waste numbers when they are actually 'over-provisioned' for very valid reasons (e.g. having to deal with frequent, but short lived spikes).

This initiative could be rolled out iteratively - e.g. adding to Tamland first, but adding alerts later.