Skip to content

Add grafana dashboard for registry metrics

Sarah Yasonik requested to merge add-registry-metrics into master

Prometheus monitoring of the registry was enabled by default in omnibus-gitlab#3942 (closed). This MR adds a dashboard for container registry metrics to make those metrics more visible.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/59266

Testing

The following testing instructions can be used to see the new dashboard. Parts of these instructions will be changing in the next few weeks (once omnibus ships with grafana on by default).

Environment Setup

How: Run gitlab in docker from the omnibus package (enabling grafana and container regsitry)

  1. We're setting up the container registry for http instead of https, so temporarily add localhost:5000 to the insecure registries for docker (detailed instructions: https://docs.docker.com/registry/insecure/).
  2. Create a temporary directory for logs/config/etc. Replace /Users/syasonik/workspace/gitlab-docker-tmp on line 4 in the script below with the path to your directory.
  3. Use the below script to start the docker container. It's based on https://gitlab.com/snippets/1832741, but also enables grafana, enables the registry, disables ssl, and exposes the right ports to access the registry.
set -uo pipefail

image='gitlab/gitlab-ce:nightly'
tmpdir='/Users/syasonik/workspace/gitlab-docker-tmp'

usage() {
  cat << USAGE
usage: $(basename $0) <opts>
  -h          Display this message
  -i <image>  Docker image (Default "${image}")
  -t <dir>    Temporary directory for running (Default /tmp/gitlab-XXXXXX)
USAGE
}

opt='' OPTIND='1' help='0'
while getopts 'hi:t:' opt ; do
  case "${opt}" in
    h) help='1' ;;
    i) image="${OPTARG}" ;;
    t) tmpdir="${OPTARG}" ;;
    *) 
      echo 'ERROR: Invalid flag'
      usage 
      exit 1 ;;
  esac
done
shift $((OPTIND - 1))

if [[ ${help} -ne 0 ]] ; then
  usage
  exit
fi

if [[ ! -d "${tmpdir}" ]] ; then
  tmpdir=$(mktemp -d /tmp/gitlab-XXXXXX)
fi

echo "Temp dir: ${tmpdir}"

mkdir -p "${tmpdir}/config"
mkdir -p "${tmpdir}/logs"
mkdir -p "${tmpdir}/data"

docker run \
  --env GITLAB_OMNIBUS_CONFIG="grafana['enable'] = true; grafana['allow_user_sign_up'] = true; registry_external_url 'http://localhost:5005'; letsencrypt['enable'] = false; registry['token_realm'] = 'http://localhost'; registry['debug_addr'] = 'localhost:5001';" \
  --publish 443:443 --publish 80:80 --publish 22:22 --publish 5005:5005 --publish 5001:5001 \
  --name gitlab \
  --restart always \
  --volume "${tmpdir}/config:/etc/gitlab" \
  --volume "${tmpdir}/logs:/var/log/gitlab" \
  --volume "${tmpdir}/data:/var/opt/gitlab" \
  "${image}"

Verification

  1. Create a project.
  2. Interact with the container registry.
  3. See the dashboard with data in grafana.
Interact with the container registry.

Note: These instructions follow with the docs on the /registry page of your project. Just some extra detail is included below based on what I did.

  1. Create a personal access token.
  2. Login to the registry, using your personal access token as the password and root as the username.
docker login localhost:5005
Username: root
Password: <personal access token>
  1. Create a directory with generic dockerfile - doesn't matter what's in it really. cd into the directory. I just downloaded https://gitlab.com/tjamet/docker-compose out of laziness.
  2. Build the image with docker build -t localhost:5005/root/docker-compose .
  3. Push the image with docker push localhost:5005/root/docker-compose
See the dashboard with data in grafana.
  1. Navigate to localhost/~/grafana/ (Note, it may take some time to become available. Wait a few minutes.)
  2. Login with username admin/admin.
  3. Either Skip the password update or change the grafana password.
  4. Add the file in this branch as a new dashboard.
  5. See all the great stuff there!

Screen_Shot_2019-06-03_at_6.43.03_PM

Edited by 🤖 GitLab Bot 🤖

Merge request reports