GitLab-CI: tracking artifacts by commit
Description
In general, I would like to deploy the same artifact to each environment instead of rebuilding them. The flow would look something like this, when an artifact is built from the "dev" branch and deployed to Development that build is kept track of in relation to the commit. Then when that commit is merged to into the "qa" branch and deploy to QA, it should be the same exact artifact without rebuilding anything. This absolutely guarantees that what is going to the higher environment has been tested in the lower environment. This is currently achievable a couple of different ways, none of which are particularly ideal.
This feature also shouldn't be tied to GitLab Registry. It shouldn't be limited to building docker images when GitLab-CI can create any artifact it wants and add it to a Release (https://docs.gitlab.com/ee/api/projects.html#upload-a-file).
Current approach: Utilizing GitLab Registry, the CI process creates a docker image with the tag of the SHA. Using the fast-forward merge method, the build process then can pull down the image from the sha and retag is as a QA built thus ensuring it's the same artifact from the lower environment. This then also prevents anyone from making untested changes directly into QA. I'll put an example gitlab-ci.yml of this in the blurb section.
concerns:
- Using fast-forward seems like a hack
- create a hotfix would require a different flow (though could be accounted for in the .gitlab-ci.yml file).
- limited to Docker (can't use the Releases feature)
Environment approach: I also ran into the documentation around environments in GitLab-CI and how to do this: https://docs.gitlab.com/ce/ci/environments.html This would achieve the same thing, however it wouldn't be an automatic deployment to the higher environments.
concerns:
- no automatic deployment to each environment
- limited to Docker
Proposal
Add a feature that somehow can keep track of what resulted from a build process so that when the code is merged into subsequent branches the GitLab-CI process can skip the build.
Links / references
Documentation blurb
stages:
- build
- release
before_script:
- export VER=`cat config/VERSION`
- export NAME=$CI_REGISTRY_IMAGE/$VER:$CI_COMMIT_SHA
build:
image: docker:latest
services:
- docker:dind
stage: build
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build -t $NAME .
- docker push $NAME
only:
- dev
release:
image: docker:latest
services:
- docker:dind
stage: release
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- export PRODNAME=$CI_REGISTRY_IMAGE/$VER:production
- docker pull $NAME
- docker tag $NAME $PRODNAME
- docker push $PRODNAME
only:
- master
Overview
What is it? Use same artifacts instead of rebuilding for each branch
Why should someone use this feature?
- save build time costs
- save storage costs (if artifacts are large)
- help provide consistency between environments.
What is the underlying (business) problem? Ensuring what is going to production has been properly tested in lower environments
How do you use this feature? In GitLab-CI for CD
Use cases
As a developer, when my code is build in dev to create an artifact I want my subsequent environments (qa,stage,prod,etc) to refer to that artifact and use it for deployment instead of creating a new one.
Feature checklist
-
Track artifacts through branches to avoid rebuilding and always using the same artifact for deployment -
should support both GitLab Registry and the Release artifacts