Skip to content

Draft: Add auto-generated Cue schema for `.gitlab-ci.yml` PoC

Grzegorz Bizon requested to merge feature/gb/poc-gitlab-ci-cue into master

Description

This merge request demonstrates that we can generate Cue configuration from .gitlab-ci.yml in an automatic way. We can later convert this into JSON Schema or to make it a base of extending complex configs that then can be converted back to YAML.

This is a naive and simplistic PoC. I took a random config from our codebase, and made it possible to generate Cue for it. Currently it only works for:

image: alpine:latest

variables:
  DB_NAME: postgres

stages:
  - test
  - deploy
  - notify

staging:
  image: ruby:alpine
  variables:
    KEY1: value1
    KEY2: value2
  script: "cap deploy stating"
  stage: deploy
  tags:
    - ruby
    - mysql

You can generate Cue schema for it using following code snippet:

yml = File.read('spec/support/gitlab_stubs/gitlab-ci-simple.yml')

puts Gitlab::Ci::Config.new(yml).root.to_cue

Note: it currently only works for this YAML content, we would need to describe Cue schema for remaining types / CI keywords, but as the PoC shows it seems to be completely feasible to do that.

After you write it to a file, let's say /tmp/gitlab-ci.cue, you can run cue fmt /tmp/gitlab-ci.cue. Following content will be the output:

#Image: string | {
	name:       string
	entrypoint: string
	ports:      string
}

#Variables: [=~"^[A-Z0-9._-]+$"]: string

#Stages: [...string]

#Stage:    string
#Commands: string | [...string]
#Tags: [...string]

image:     #Image
variables: #Variables
stages:    #Stages

[=~"^[a-zA-Z0-9._-]+$" & !~"^(image|stages|variables)$"]: #Job

#Job: {
	stage:     #Stage
	variables: #Variables
	script:    #Commands
	image:     #Image
	tags:      #Tags
}

Comment: As you can see we are already benefiting from Cue config order independence

This can be then used to validate the source YAML:

cue vet spec/support/gitlab_stubs/gitlab-ci-simple.yml /tmp/gitlab-ci.cue

From here it would be attainable (medium effort) to convert .gitlab-ci.yml to .gitlab-ci.cue completely, in the other direction, and to have a complete schema definition in Cue, describing everything that you can do in CI configuration.

/cc @marshall007 @dhershkovitch

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Grzegorz Bizon

Merge request reports