Operator on ppc64/s390x

Related to: gitlab-org/gitlab-runner#2941 (closed)

I went on and compiled a walkthrough of how to build everything needed for the operator for ppc64. The approach should be the same for x390. I have no ppc64 OpenShift cluster to test this but in general it should work. I am a bit doubtful about the way the manifest is tagged at the end but opm doesn't seem to support that out of the box, unlike the legacy tool oc adm: https://docs.openshift.com/container-platform/4.5/operators/admin/olm-managing-custom-catalogs.html

This should serve as a good guide on how to get multiarch support in all places of the pipeline for when we decide to do it.

  • GitLab Runner (we already have that)
  • UBI Images
  • Operator Image
  • Operator Bundle
export TINI_VERSION=0.18.0
export OS=linux
export ARCH=ppc64le
export GITLAB_RUNNER_UBI_IMAGES_REPO_PATH="SET_THIS"
# Enable docker buildx: https://docs.docker.com/buildx/working-with-buildx/
# Make sure your docker host has qemu support, if using MacOS it should work out of the box
# For Linux follow instructions for your distribution, e.g. for Ubuntu:
# sudo apt-get install -y emu binfmt-support qemu-user-static qemu-system-ppc64
# docker run --rm --privileged multiarch/qemu-user-static --reset -p yes 
export DOCKER_CLI_EXPERIMENTAL=enabled
export BASE_IMAGE=registry.access.redhat.com/ubi8/ubi:8.4-203
export VERSION=v13.12.0
export GIT_LFS_VERSION=2.11.0-2.el8
# Replace with your docker registry address and username
export DOCKER_REGISTRY="docker.io/gngeorgiev"

#### In https://gitlab.com/gitlab-org/ci-cd/gitlab-runner-ubi-images (Commit: e265820a00a6a1b9a271dc132de2618ced43cf92)

./ubi.sh prepare "${VERSION}"
curl -Lf "https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini-static-${ARCH}" -o ./build/runner/tini

#### In https://gitlab.com/gitlab-org/gitlab-runner (Commit: 439ad8853ef5adec154bc4eb15e54c04a34ee15f)

# BEWARE: This will create a dev build from the current checked out commit
GOOS="${OS}" GOARCH="${ARCH}" go build -o "./binaries/gitlab-runner.${ARCH}" .
GOOS="${OS}" GOARCH="${ARCH}" go build -o "./binaries/gitlab-runner-helper.${ARCH}" ./apps/gitlab-runner-helper/

mv "./binaries/gitlab-runner.${ARCH}" "${GITLAB_RUNNER_UBI_IMAGES_REPO_PATH}/build/runner/gitlab-runner"
mv "./binaries/gitlab-runner-helper.${ARCH}" "${GITLAB_RUNNER_UBI_IMAGES_REPO_PATH}/build/helper/gitlab-runner-helper"

#### Back in https://gitlab.com/gitlab-org/ci-cd/gitlab-runner-ubi-images (Commit: e265820a00a6a1b9a271dc132de2618ced43cf92)

# Make sure to check above points for docker buildx
# NOTE: Optionally add "--push" argument to each "docker buildx" command to push the image to the registry automatically
docker buildx build --platform "${OS}/${ARCH}" \
    --build-arg BASE_IMAGE="${BASE_IMAGE}" \
    --build-arg VERSION="${VERSION}" \
    --build-arg GIT_LFS_VERSION="${GIT_LFS_VERSION}" \
    -f ./build/runner/Dockerfile.OCP \
    -t "${DOCKER_REGISTRY}/gitlab-runner-ocp:${VERSION}" ./build/runner

docker buildx build --platform "${OS}/${ARCH}" \
    --build-arg BASE_IMAGE="${BASE_IMAGE}" \
    --build-arg VERSION="${VERSION}" \
    --build-arg GIT_LFS_VERSION="${GIT_LFS_VERSION}" \
    -f ./build/helper/Dockerfile.OCP \
    -t "${DOCKER_REGISTRY}/gitlab-runner-helper-ocp:${ARCH}-${VERSION}" ./build/helper

#### In https://github.com/brancz/kube-rbac-proxy

git checkout v0.5.0
docker buildx build --platform "${OS}/${ARCH}" \
    --build-arg BINARY="kube-rbac-proxy" \
    -t "${DOCKER_REGISTRY}/kube-rbac-proxy:v0.5.0" .


#### In https://gitlab.com/gitlab-org/gl-openshift/gitlab-runner-operator (Commit: d59a375)

git checkout multiarch-support

CERTIFIED=false \
    ARCH="${ARCH}" \
    UPSTREAM_UBI_IMAGES_REPOSITORY="${DOCKER_REGISTRY}" \
    ./scripts/create_release_config.sh v1.0.0 "${VERSION}"

docker buildx build --platform "${OS}/${ARCH}" \
    --build-arg VERSION="${VERSION}" \
    --build-arg GOARCH="${ARCH}" \
    -f Dockerfile \
    -t "${DOCKER_REGISTRY}/gitlab-runner-operator-${OS}-${ARCH}" .

CERTIFIED=false \
    ARCH="${ARCH}" \
    UPSTREAM_UBI_IMAGES_REPOSITORY="${DOCKER_REGISTRY}" \
    GITLAB_RUNNER_OPERATOR_REGISTRY="${DOCKER_REGISTRY}" \
    KUBE_RBAC_PROXY_IMAGE="${DOCKER_REGISTRY}/kube-rbac-proxy:v0.5.0" \
    ./scripts/create_related_images_config.sh v1.0.0 "${VERSION}"

make certification-bundle

docker buildx build --platform "${OS}/${ARCH}" \
    --build-arg VERSION="${VERSION}" \
    -f certification.Dockerfile \
    -t "${DOCKER_REGISTRY}/gitlab-runner-operator-bundle:v1.0.0" .

opm index add --bundles "${DOCKER_REGISTRY}/gitlab-runner-operator-bundle:v1.0.0" \
    --from-index quay.io/operatorhubio/catalog:latest \
    --tag "${DOCKER_REGISTRY}/operator-catalog:v1.0.0" \
    --build-tool docker

docker manifest create "${DOCKER_REGISTRY}/operator-catalog:v1.0.0" "${DOCKER_REGISTRY}/operator-catalog:v1.0.0"
docker manifest annotate --os="${OS}" --arch="${ARCH}" "${DOCKER_REGISTRY}/operator-catalog:v1.0.0" "${DOCKER_REGISTRY}/operator-catalog:v1.0.0"
docker manifest push "${DOCKER_REGISTRY}/operator-catalog:v1.0.0"

cat > catalogsource.yaml << EOF
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: gitlab-runner-catalog
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: ${DOCKER_REGISTRY}/operator-catalog:v1.0.0
  displayName: GitLab Runner Operators
  publisher: GitLab Community 
EOF

oc apply -f catalogsource.yaml
Edited by Elliot Rushton