Epic: CI/CD Golden Component Migration
## Overview
**Epic:** CI/CD Golden Component Migration
**Status:** Active Development
**Priority:** P0 - Blocking 60+ projects
**Milestone:** v0.3.x
## Mission
Migrate all 60+ projects in the Agent Platform to the gitlab_components 2-line pattern. Build component catalog, versioning, and orchestration layer. Validate 109 templates. Execute Wave 1-4 migrations.
---
## Current Status: CRITICAL BLOCKER
### Active Blockers
- `spec:` section missing from golden component (Issue #447) - BLOCKING 10 MRs
- Template syntax validation errors (Issue #448) - BLOCKING 70+ projects
- Need stabilization and v0.1.5 release (Issue #331)
### Immediate Actions Required
1. Fix golden component `spec:` section
2. Audit and validate 109 template syntax
3. Create v0.1.5 stabilization release
4. Test with 5 pilot projects before rollout
---
## Architecture
### 2-Line Pattern
Every project CI/CD reduces to 2 lines:
```yaml
include:
- component: gitlab.com/blueflyio/gitlab_components/golden@~0.1
inputs:
project_type: npm
node_version: "20"
```
### Component Catalog
- **Golden Component**: Master orchestration layer
- **27 Extracted Components**: Specialized components (build, test, deploy, validate, etc.)
- **Conditional Inclusion**: Project-type aware configuration
- **Auto-versioning**: CI auto-increments patch versions
### Separation of Duties
- `gitlab_components`: Component definitions and orchestration
- Each project: 2-line `.gitlab-ci.yml` with project-specific inputs
- Zero duplication across 60+ projects
---
## Phases
### Phase 1: Component Standards & Catalog
**Goal:** Build golden component v1.0 and component catalog
**Issues:**
- #353: Audit current CI/CD adoption across 56 projects
- #354: Document Golden Component v1.0 standards
- #357: Create migration templates
- #359: Create orchestration layer
- #447: FIX spec section (CRITICAL BLOCKER)
- #448: Audit and validate template syntax (CRITICAL)
- #331: Stabilize golden component (v0.1.5 release)
- #332: Implement dynamic versioning (auto-patch increments)
- #336: Verify golden component works across 70+ projects
- #342-#347: Add conditional rules and inputs
- #350-#352: Test golden component with NPM and Drupal projects
**Success Criteria:**
- ✅ Golden component v0.1.5 released and stable
- ✅ 109 templates validated
- ✅ Component catalog published
- ✅ Documentation complete
- ✅ Migration templates ready
---
### Phase 2: Wave 1 - Core Platform Projects (5 projects)
**Goal:** Migrate critical infrastructure projects
**Issues:**
- #364: Wave 1 migration
**Projects:**
- platform-agents
- openstandardagents
- agent-buildkit
- agent-router
- agent-protocol
**Success Criteria:**
- ✅ All 5 projects migrated
- ✅ CI/CD passing
- ✅ No performance regression
---
### Phase 3: Wave 2 - NPM Packages (12 projects)
**Goal:** Migrate NPM package projects
**Issues:**
- #365: Wave 2 migration
**Projects:**
- agent-brain
- agent-chat
- agent-docker
- agent-mesh
- agent-studio
- agent-tracer
- agent_tailscale
- agentic-flows
- compliance-engine
- foundation-bridge
- workflow-engine
- studio-ui
**Success Criteria:**
- ✅ All 12 projects migrated
- ✅ NPM-specific templates working
---
### Phase 4: Wave 3 - Infrastructure & Tools (8 projects)
**Goal:** Migrate infrastructure and tooling projects
**Issues:**
- #366: Wave 3 migration
**Projects:**
- gitlab_components (self-migration)
- api-schema-registry
- technical-docs
- project-template
- security-policies
- kagent
- openstandardagents.org
- demo projects
**Success Criteria:**
- ✅ All 8 projects migrated
- ✅ Self-migration successful
---
### Phase 5: Wave 4 - Drupal & Models (16 projects)
**Goal:** Migrate Drupal modules and model training projects
**Issues:**
- #368: Wave 4 migration
**Projects:**
- All Drupal custom modules (16 projects)
- Model training projects
**Success Criteria:**
- ✅ All Drupal projects migrated
- ✅ Drupal-specific templates working
---
## Key Features
### Golden Component Capabilities
- **Auto-detection**: Detects project type from manifest/package.json
- **Conditional Inclusion**: Includes only relevant components per project type
- **Environment-aware**: Adapts to development, staging, production
- **GitLab Ultimate Integration**: SAST, DAST, dependency scanning, secret detection
- **Observability**: OpenTelemetry tracing, error tracking, performance monitoring
- **Compliance**: SOC 2, GDPR, HIPAA frameworks
### Supported Project Types
- NPM packages
- Drupal modules
- Python packages
- Infrastructure (Terraform, K8s)
- Documentation sites
- API services
---
## Validation & Testing
### 109 Template Validation (Issue #448)
- Audit all 109 templates in `templates/` directory
- Fix syntax and validation errors
- Ensure `spec:` sections defined
- Test with pipeline editor
### Pilot Testing
- Test with 5 pilot projects before wave rollout
- Verify CI/CD passing
- Monitor performance
- Collect feedback
---
## Migration Strategy
### Pre-migration
1. Audit current CI/CD setup
2. Document dependencies
3. Identify blockers
4. Create rollback plan
### Migration
1. Update `.gitlab-ci.yml` to 2-line pattern
2. Configure inputs for project type
3. Remove old CI/CD code
4. Test in MR
5. Merge when green
### Post-migration
1. Monitor CI/CD performance
2. Track metrics (duration, cost, success rate)
3. Document lessons learned
4. Update migration templates
---
## Success Metrics
### Component Adoption
- **Target:** 100% of 60+ projects migrated
- **Current:** ~10% (6 projects)
- **Blocker:** spec section and validation errors
### CI/CD Efficiency
- **Target:** 30% reduction in pipeline duration
- **Target:** 50% reduction in duplicated CI/CD code
- **Target:** 95%+ migration success rate
### Quality
- **Target:** All projects passing CI/CD
- **Target:** Zero security vulnerabilities
- **Target:** Compliance framework implemented
---
## Dependencies
### Prerequisites
- GitLab Ultimate features enabled
- Component catalog accessible
- OIDC tokens configured
- Observability infrastructure deployed
### Blockers
- Issue #447: spec section missing (CRITICAL)
- Issue #448: Template validation errors (CRITICAL)
- Issue #331: Stabilization needed
---
## Timeline
**Week 1-2:** Phase 1 - Fix blockers, stabilize component
**Week 3-4:** Wave 1 - Core platform (5 projects)
**Week 5-6:** Wave 2 - NPM packages (12 projects)
**Week 7-8:** Wave 3 - Infrastructure (8 projects)
**Week 9-10:** Wave 4 - Drupal & Models (16 projects)
**Total:** 10 weeks to complete all migrations
---
## Related Epics
- #54: Ultimate Agent Platform v1.0 - Production Launch
- #55: GitLab Ultimate Showcase Initiative
- #18: Agent Consolidation (150→30 agents)
---
## Resources
- [gitlab_components Wiki](https://gitlab.com/blueflyio/gitlab_components/-/wikis/home)
- [Golden Component Documentation](https://gitlab.com/blueflyio/gitlab_components/-/wikis/golden-component)
- [Migration Guide](https://gitlab.com/blueflyio/gitlab_components/-/wikis/migration-guide)
- [Component Catalog](https://gitlab.com/blueflyio/gitlab_components/-/blob/main/templates/)
---
**Owner:** Thomas Scola (@tscola)
**Group:** blueflyio
**Milestone:** v0.3.x
**Priority:** P0 - Critical
epic