Skip to content

Add gitlab-gkg CNG image

Dmitry Gruzdrequested to merge
add-gitlab-gkg-image into master

Overview

This MR introduces a new CNG component gitlab-gkg that containerizes the GitLab Knowledge Graph service, providing dual-mode operation similar to gitlab-zoekt. The image supports both indexing and querying workloads through a single container with configurable modes.

What does this MR do?

  • Adds new CNG component: gitlab-gkg image with dual-mode architecture
  • Uses pre-built binaries: Downloads official releases from GitLab (no compilation needed)
  • Integrates with build system: Follows CNG patterns for versioning and CI/CD
  • Provides containerized Knowledge Graph: Ready-to-deploy service for GitLab environments
  • Enables flexible deployment: Single image supports both indexer and webserver modes

Architecture

Dual-Mode Operation

  • Indexer Mode (GITLAB_GKG_MODE=indexer): Runs on port 3333, handles data indexing
  • Webserver Mode (GITLAB_GKG_MODE=webserver): Runs on port 3334, serves queries
  • Configuration: Mode controlled via environment variables, similar to gitlab-zoekt

Technical Stack

  • Base Image: gitlab-base (both builder and runtime stages)
  • Binary Distribution: Pre-built gkg-server binaries from GitLab releases
  • Version: v0.20.0 (starting from this version, releases include pre-built binaries)
  • Architecture Support: x86_64 and aarch64 (arm64)
  • Authentication: JWT-based with configurable secret path
  • Health Monitoring: Built-in health checks for both modes
  • Data Persistence: /data/gkg volume for persistent storage

Key Files

New Components

gitlab-gkg/
├── Dockerfile                 # Multi-stage build with pre-built binary download
├── scripts/
│   ├── process-wrapper       # Mode selection and process execution
│   └── healthcheck          # Health monitoring for both modes

Build System Integration

  • ci_files/variables.yml: Added GITLAB_GKG_SHA/GITLAB_GKG_TAG variables
  • build-scripts/container_versions.sh: Added get_gkg_version() function
  • .gitlab/ci/common.gitlab-ci.yml: Added .gitlab-gkg build template
  • .gitlab/ci/debian.gitlab-ci.yml: Added gitlab-gkg job configuration

Configuration

Environment Variables

Variable Description Default
GITLAB_GKG_MODE Operating mode (indexer | webserver) Required
GITLAB_GKG_SECRET_PATH JWT secret file path /.gitlab_shell_secret
GITLAB_GKG_DATA_DIR Data directory /data/gkg
PORT Override default port 3333/3334 by mode

Build Arguments

Argument Description Default
GITLAB_GKG_SHA Release version tag for pre-built binary v0.20.0

Usage Examples

Indexer Mode

docker run -d \
  -e GITLAB_GKG_MODE=indexer \
  -e GITLAB_GKG_SECRET_PATH=/secrets/jwt-secret \
  -v /data/gkg:/data/gkg \
  -p 3333:3333 \
  registry.gitlab.com/gitlab-org/build/cng/gitlab-gkg:latest

Webserver Mode

docker run -d \
  -e GITLAB_GKG_MODE=webserver \
  -e GITLAB_GKG_SECRET_PATH=/secrets/jwt-secret \
  -v /data/gkg:/data/gkg \
  -p 3334:3334 \
  registry.gitlab.com/gitlab-org/build/cng/gitlab-gkg:latest

Technical Considerations

Binary Distribution Strategy

  • Approach: Download pre-built binaries from GitLab releases (available since v0.20.0)
  • Benefits: Eliminates Rust toolchain installation and compilation, significantly faster builds
  • Security: Checksums verified using SHA256 signatures from release artifacts
  • No Build Dependencies: Removed gitlab-rust dependency from CI pipeline

Build Performance

  • Multi-stage optimization: Builder stage downloads and extracts binary, runtime stage runs it
  • Fast downloads: ~7MB compressed binary (vs. 40+ seconds compilation time)
  • Size efficiency: Final image based on minimal gitlab-base with only runtime dependencies
  • Simplified CI: No Rust toolchain management or version coordination needed

Security & Best Practices

  • Non-root execution: Runs as git user following CNG patterns
  • Secret management: JWT authentication via mounted secrets
  • Health monitoring: Built-in endpoints for both operational modes
  • Data persistence: Proper volume handling for stateful data

Testing

The image has been successfully built and tested:

  • ✅ Multi-stage build completes (~3 seconds download time)
  • ✅ Binary downloaded and checksum verified successfully
  • ✅ Binary executes correctly (gkg-server v0.20.0)
  • ✅ Health check scripts execute properly
  • ✅ Both operational modes supported
  • ✅ Tested on x86_64 and arm64 architectures

Migration & Rollout

This is a new component addition with no breaking changes to existing services.

Deployment Readiness

  • Follows established CNG patterns for consistency
  • Integrates with existing build pipeline
  • Compatible with GitLab's container orchestration
  • Ready for staged rollout across environments

Related Issues

Closes gitlab-org/rust/knowledge-graph#184

Checklist

  • Image builds successfully using CNG build system
  • Follows CNG architectural patterns (multi-stage, security practices)
  • Integrates with version management and CI/CD pipeline
  • Includes proper documentation and configuration examples
  • Health checks implemented for both operational modes
  • Tested locally with successful build completion
Edited by Dmitry Gruzd

Merge request reports