Skip to content

Introduce builder images

Hossein Pursultani requested to merge hp-builder-images into master

What does this MR do?

Overview

This MR contains:

  1. core and go builder images. core is a base build environment with essential build tools, such as make and gcc, development libraries and headers, and helper scripts. go is based on core builder and adds Go compiler to the mix.
  2. BUILDER_PIPELINE which builds different flavors of core and go builder images and allows verifying them by triggering typical pipelines and passing their reference.

Some definitions

  • flavor i.e. Debian, UBI, FIPS
  • family is the category of the image, i.e. builders, assets, runtime (this is further expanded in gitlab-org/charts/gitlab#4217).
  • flavored Dockerfile is a Dockerfile that is suffixed with the flavor, e.g. Dockerfile.fips. When a flavored Dockerfile does not exist the default Dockerfile is used. For fips flavor, before falling back to the default Dockerfile the UBI-flavored Dockerfile is preferred.

Builder images

core image contains:

  1. All the essential build tools
  2. All development headers and build dependencies that are required in other CNG Docker images
  3. Utility scripts including scripts for downloading and verifying archives and artifacts.

go builder image is based on core image and adds Go language compiler and tools.

The core contains a few helper scripts which can be used in the build stage of other CNG images:

  • fetch: download and extract an archive file
  • gitlab-fetch: download and extract repository archive from GitLab
  • apply-patches: populates and applies the specified patches
  • cleanup-eggs: removes Python packages clutter
  • cleanup-gems: removes Ruby Gems clutter
  • detect-distro: detects Linux distribution, version, and architecture

Builder pipeline

The BUILDER_PIPELINE type is limited to the changes to builders/ directory. This directory is intended to group together the images that belong to the builder family.

This pipeline, which is supposed to only run on master and MR branches of .com repository, builds the different flavors of core and go builder images and add manual triggers for verifying the images. Each flavor has its own trigger. The triggers pass the reference the corresponding builder images of the flavor that are built for the current branch to the downstream pipelines.

Note that the build scripts of this pipeline are not added to build.sh and are written in a verbose style. This is done intentionally in order to allow further improvements to build.sh.

Related issues

Related to:

  1. gitlab-org/charts/gitlab#4216
  2. gitlab-org/charts/gitlab#3444 (moved)
  3. gitlab-org/charts/gitlab#1257 (moved)
  4. gitlab-org/charts/gitlab#3480
  5. gitlab-org/charts/gitlab#3450 (moved)
  6. gitlab-org/charts/gitlab#2439 (moved)
  7. gitlab-org/charts/gitlab#3407 (moved)

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion

Required

  • Merge Request Title, and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • Documentation created/updated
  • Integration tests added to GitLab QA
  • The impact any change in container size has should be evaluated
Edited by Jason Plum

Merge request reports