Key Value Store for Version Infos of CI/CD Pipeline

Problem to solve

Our scenario

Our CI/CD Pipelne has to manipulate versions on specific places, eg

  • Set the version into our gradle files
  • Set the version into our Dockerfiles
  • Set the version into our k8s.yaml files
  • ...

So there has to be a Single Source of Truth for the VersionInfo during a build. A build can take hours and paralell builds of the same thing can happen (with the latest build having a newer Version)

Our old solution

Using Git Tags => Did not work out due to with > 75000 Tags some things get slow

Our new approach

Having a version file in each repo which gets incremented and pushed at the start of each pipeline. Within the pipeline run the version stays the same.

Due to doing git pushes within a pipeline also has some drawbacks we consider using an external DB for the VersionInfo

Target audience

Architects of Build Pipelines

Further details

Proposal

Gitlab could offer a KeyValueStore where the VersionInfo is stored.

Storing Data

Key = {"Pipeline":"89D95479","Artifact":"My-App"} Value = 2.0.9

Getting Data

curl https://gitlab.com/api/v4/projects/9/pipeline/89D95479?artifact=My-App

What does success look like, and how can we measure that?

The Gitlab Rest API allows POST and GET calls which can be used within .gitlab-ci.yaml files

Links / references