Publish docker images to ECR registry

Overview

With Amazon's new registry we want to publish our docker images to that registry as well as the gitlab one and dockerhub.

The account to push this to is already set up and configured. We should just need to login and push.

CC @DarrenEastman

Resources

Getting started

GitLab Namespace

Both k2y5y2m7 and gitlab are valid aliases.

describe-registries

 ecr-public-beta aws --region us-east-1 --profile distribution ecr-public describe-registries
{
    "registries": [
        {
            "registryId": "782774275127",
            "registryArn": "arn:aws:ecr-public::782774275127:registry/782774275127",
            "registryUri": "public.ecr.aws/gitlab",
            "verified": true,
            "aliases": [
                {
                    "name": "k2y5y2m7",
                    "status": "ACTIVE",
                    "primaryRegistryAlias": false,
                    "defaultRegistryAlias": true
                },
                {
                    "name": "gitlab",
                    "status": "ACTIVE",
                    "primaryRegistryAlias": true,
                    "defaultRegistryAlias": false
                }
            ]
        }
    ]
}

GitLab Runner repository

GitLab Runner helper repository

Authentication

Download onboarding zip
Follow the README.pdf

Setting up authentication for CI

Create IAM role for CI with the least privileged to authenticate with the registry to be able to push images.
For every job run the aws --region us-east-1 ecr-public get-authorization-token --output=text --query 'authorizationData.authorizationToken' | base64 --decode | cut -d: -f2 to get a docker login. According to get-authorization-token this token expires after 12hours so we have to generate one for every job to make sure we never get an expired token when trying to push.
Run docker logging from the output of the previous command, for example aws --region us-east-1 ecr-public get-authorization-token --output=text --query 'authorizationData.authorizationToken' | base64 --decode | cut -d: -f2 | docker login -u AWS --password-stdin https://public.ecr.aws

Mirror images

At launch, we want to have the latest gitlab/gitlab-runner and gitlab/gitlab-runner-helper images published already in the ECR registry for users to already use

Mirror `gitlab/gitlab-runner`

You can use the script below to mirror any tag we need, for example,./mirror.sh gitlab/gitlab-runner:alpine

mirror.sh

```bash
#!/usr/bin/env bash

# gitlab/gitlab-runner:latest
image=$1
tag=$(echo $1 | sed 's/.*://')
imageName=$(echo $1 | sed 's/:.*//')
ecrRegistry=public.ecr.aws
manifest=""

archs=("amd64" "arm64" "s390x")

for arch in "${archs[@]}"
do
    echo "arch: $arch"
    docker pull --platform "$arch" "$image"
    docker tag "$image" "$ecrRegistry/$imageName:$tag-$arch"
    docker push "$ecrRegistry/$imageName:$tag-$arch"
    docker rmi "$image"
    manifest+=" --amend $ecrRegistry/$imageName:$tag-$arch"
done

docker manifest create "$ecrRegistry/$imageName:$tag" $manifest
docker manifest push "$ecrRegistry/$imageName:$tag"

Mirror `gitlab/gitlab-runner-helper`

Linux

Usage: ./mirror-helper.sh latest

mirror-helper.sh

#!/usr/bin/env bash

# 8fa89735
tag=$1
ecrImage="public.ecr.aws/gitlab/gitlab-runner-helper"
dockerhubImage="gitlab/gitlab-runner-helper"

archs=("x86_64" "arm64" "s390x" "arm")

for arch in "${archs[@]}"
do
  docker pull "$dockerhubImage:$arch-$tag"
  docker tag "$dockerhubImage:$arch-$tag" "$ecrImage:$arch-$tag"
  docker push "$ecrImage:$arch-$tag"
done

Windows

Usage: pwsh -f .\mirror-helper.pwsh -tag latest-servercore1809

mirror-helper.pwsh

param([String]$tag="")

$dockerhubImage="gitlab/gitlab-runner-helper"
$ecrImage="public.ecr.aws/gitlab/gitlab-runner-helper"

docker login https://public.ecr.aws
docker pull "${dockerhubImage}:x86_64-${tag}"
docker tag "${dockerhubImage}:x86_64-${tag}" "${ecrImage}:x86_64-$tag"
docker push "${ecrImage}:x86_64-${tag}"

Edited Dec 02, 2020 by Steve Azzopardi