Further ideas to reduce cloud costs
-
create/improve Grafana dashboard to visualize build job duration; use DW data if possible, and divide by rpm/upstream, rhel version, ... -
investigate ccache coverage, cache hit/miss penalty, and its effect on build time. -
investigate effect of CPU count, CPU frequency -
Determine the relationship between IOPS, throughput of storage, on duration of the build, experiment with placing sources into tmpfs. -
Change the EBS volume type from gp2 to gp3, to have 20% storage cost saving realized with increased read IOPS: https://gitlab.cee.redhat.com/cki-project/deployment-all/-/merge_requests/2405 . Launch templates updated, newly deployed instances will use the new volume type. -
Investigate possibility of launching builds in other EC2 regions, depending on spot price of the instance. -
Measure and optimize cost per unit (cost per build in case of kernel build). -
Install CloudWatch agent to better understand instance utilization, including disk utilization. -
investigate s3 transfer costs: pipeline-definition!1702 (merged) -
create EC2 machine reaper: cki-tools!741 (merged) -
create S3 artifact cleaner: #95, #107 (closed) -
tag shared cluster: deployment-all!1647 (merged) -
Change the frequency of data collection for AWS Config. As AWS Config is IT-managed, I requested the improvement in https://redhat-internal.slack.com/archives/C04LX6S77NE/p1705586996872539 .
Edited by Bohdan Khomutskyi