Efficient AI DevSecOps for Ops/Sec: CI/CD, GitOps, Kubernetes, Observability: AI-assisted Auto DevOps (Auto DevOps++)

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Release notes

Problem to solve

Modern software development does not happen in monorepos anymore. The projects are spanned across groups, use different programming languages and build tools, are deployed as microservices, and use GitOps patterns with the agent for Kubernetes. All of this requires combining the different sources into one central pipeline, where Auto DevOps can help with automated build and deploy workflows.

Efficient pipelines require manual addition after detecting large container images, many repeated cycles of dependency installs (missing caching), etc.

Proposal

Think of a step-by-step process that implements the stages of Auto DevOps, with the most efficient actions automatically applied, and helped with AI knowledge of which pattern works best for the given project(s).

1. Auto-discover involved projects and dependencies in a group of projects.
2. Propose multiple CI, a single CD, or otherwise best practice pipelines (parent-child, multi project, triggers)
3. Identify common best practice patterns (efficiency with matrix builds, caching, optimized container images, etc.)
4. Apply security scanning based on all projects, and use AI to avoid duplicates and filter out noise that makes engineers fatigue
5. Integrate agent for Kubernetes, GitOps/Flux with opinionated best practice deployments into Kubernetes
6. Include infrastructure provisioning
7. Enable SLSA and SBOM generation by default, and provide artifacts
8. Add Observability for metrics/traces/logs/profiling/etc. (when available, non-blocking)
9. Fast feedback in MRs: Observe the Auto DevOps runs, and recommend optimization for easier team onboarding, better observability insights, and cost optimization (related to Treat MR integration sources as unified Observa... (#378841) and Pipeline Efficiency: Observability and AI Optim... (&11050 - closed))

Optional, but not a direct dependency

1. Add CI/CD components into the building blocks
2. Add CI Steps architecture
3. Use the same methods for AI-assisted CI/CD pipeline editing experience

The idea came up in a session at DevOps Camp Nuremberg 2023, about AI in the DevOps lifecycle, where we discussed CI/CD efficiency for some minutes. Additional resource: DevRel Event results slide deck (internal).

Intended users

• Delaney (Development Team Lead)
• Priyanka (Platform Engineer)
• Sidney (Systems Administrator)
• Allison (Application Ops)
• Amy (Application Security Engineer)
• Isaac (Infrastructure Engineer)

Feature Usage Metrics

Auto DevOps++ (or however the feature name will be) provides usage statistics by logging how often it is invoked, and which components work OOTB.

Does this feature require an audit event?

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.