<!-- This template is a great use for issues that are feature::additions or technical tasks for larger issues.--> ## Overview Pipeline efficiency is an ongoing topic. There are many ways to analyse, refactor, and optimize pipelines to reduce cost, and also leverage AI already to do forecast planning. ### Proposal <!-- Use this section to explain the feature and how it will work. It can be helpful to add technical details, design proposals, and links to related epics or issues. --> This proposal involves multiple ideas and MVCs 1. Collect CI/CD Observability data (storage, CPU, etc.) as base metrics, job logs, add tracing for CI/CD Observability (https://gitlab.com/gitlab-org/gitlab/-/issues/338943) - Data from Runner fleet management gitlab~11448658 and ~"group::observability" 1. Train AI models that help optimize pipelines with a focus on cost optimization - Fixing pipelines fast is covered in this proposal: https://gitlab.com/gitlab-org/gitlab/-/issues/386863+ ## Resources 1. @dnsmichi talk "Observability for Efficient DevSecOps Pipelines" https://go.gitlab.com/VDAvMw - Cloudland feedback https://gitlab.com/gitlab-com/marketing/corporate_marketing/corporate-marketing/-/issues/7506#note_1443211586 1. TAMland https://go.gitlab.com/etwJmr - PromCon EU 2022: How GitLab Uses Long-Term Monitoring Data For Capacity Forecasting https://go.gitlab.com/yO6NCN 1. Coroot Cost monitoring https://github.com/coroot/coroot/releases/tag/0.15.0 1. OpenCost https://www.opencost.io/ ## Suggested Tier Paid feature, which helps with reducing costs and more efficient pipelines. gitlab~3207279 <!-- Consider adding related issues and epics to this issue. You can also reference the Feature Proposal Template (https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md) for additional details to consider adding to this issue. Additionally, as a data oriented organization, when your feature exits planning breakdown, consider adding the `What does success look like, and how can we measure that?` section. --> <!-- Label reminders Use the following resources to find the appropriate labels: - Use only one tier label choosing the lowest tier this is intended for - https://gitlab.com/gitlab-org/gitlab/-/labels - https://about.gitlab.com/handbook/product/categories/features/ -->