Allow Service project to customize CD workflow
Problem 1: CI/CD workflow isn't customizable
Currently, the entire deployment pipeline is defined in the Runway CI Tasks project. Those .gitlab-ci.ymls are maintained by SREs and shared with all of the deployment projects.
However, CD flow would be different per project. For example, a promotion to production could require deployment approvals from QA/Security engineers. Or, the application project want to run DAST on the staging environment. Or, simply they want to spin up review environment. Deployment safety features (e.g. protected environments) should be configurable at the service project.
Problem 2: Lacking multi-service support
A single project often has multiple services. For example, workhorse (Go-Lang) and rails (RoR) codebase co-exist in the same project for a good reason. The other common case would be to include both frontend and backend code in the same project. These servers are exposed to different ports e.g. workhorse => 8080, rails => 8000.
However, Cloud Run only supports a single port expose per service. This means a project needs to run multiple Cloud Run services.
Problem 3: Lacking dogfooding for CD features
Service project is not currently using DORA metrics feature. This is important metrics for measuring DevOps performance.
There are more features should be dogfooded, such as https://docs.gitlab.com/ee/ci/environments/#track-newly-included-merge-requests-per-deployment.
Proposal
Service project:
include: # Include deployment job hidden anchors
- project: 'gitlab-com/gl-infra/platform/runway/ci-tasks'
ref: main
file: 'service-project/common.yml'
build:
stage: build
scripts:
- docker build ...
deploy to review:
stage: test
extends: .deploy-to-runway # Trigger downstream pipeline
environment: review/$CI_COMMIT_REF_SLUG
rules:
- if: ...
deploy to staging:
stage: staging
extends: .deploy-to-runway # Trigger downstream pipeline
environment: staging
rules:
- if: ...
deploy to production:
stage: production
extends: .deploy-to-runway # Trigger downstream pipeline
environment: production
rules:
- if: ...