Add upstream service for container virtual registry

What does this MR do and why?

This merge request adds authentication and upstream checking functionality to container virtual registries.

Main Changes

Two-Layer OCI Authentication System

Added secure token-based authentication following the OCI Distribution Specification for connecting to external container registries. The system implements the complete authentication flow:

  • Layer 1 (Auth Discovery): Automatically discovers authentication service URLs by making requests to registry endpoints and parsing WWW-Authenticate headers - no need for users to know auth URLs like https://auth.docker.io/token. There is a follow up issue to further optimize this #563343.
  • Layer 2 (Token Exchange): Exchanges stored credentials for short-lived bearer tokens with proper scope limiting
  • Bearer Token Caching: Caches tokens for 3 minutes using Rails.cache to reduce authentication overhead
  • Universal Registry Support: Works with any OCI-compliant registry without hardcoded logic (DockerHub, GCR, ECR, ACR, Harbor, Quay, etc.)

How to set up and validate locally

1. Creating the registry and upstreams

group = Group.first # should be a top level group

# create an upstream; replace the credentials accordingly
upstream = VirtualRegistries::Container::Upstream.create!(
  group: group,
  name: 'DockerHub Test',
  url: 'https://registry-1.docker.io',
  username: 'yourusername',
  password: 'yourpassword.', 
  description: 'Test DockerHub upstream'
)

# for testing purposes, create an upstream that will fail
upstream2 = VirtualRegistries::Container::Upstream.create!(
  group: group,
  name: 'DockerHub Test',
  url: 'https://registry-1.docker.io',
  username: 'yourusername',   
  password: 'wrongpassword.',      
  description: 'Test DockerHub upstream - failcase'
)

# create the registry
registry = VirtualRegistries::Container::Registry.create!(
  group: group,
  name: 'Test Container Registry'
)

# add the upstreams to the registry
VirtualRegistries::Container::RegistryUpstream.create!(
  registry: registry,
  upstream: upstream,
  group: group,
  position: 1
)

VirtualRegistries::Container::RegistryUpstream.create!(
  registry: registry,
  upstream: upstream2,
  group: group,
  position: 2
)

When accessing a public repository, you may also test the upstream with nil credentials:

upstream.update(username: nil, password: nil)

2. Testing Upstream

a. #url_for(path)
upstream.url_for('library/alpine/manifests/latest')
# => "https://registry-1.docker.io/v2/library/alpine/manifests/latest"

upstream.url_for('nginx/nginx/tags/list')
# => "https://registry-1.docker.io/v2/nginx/nginx/tags/list"

b. #headers

upstream.headers(nil)
=> {"Authorization"=> "Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsIng1YyI6W.....`"}

upstream2.headers(nil)
=> {}

2. Testing CheckUpstreamsService

service = VirtualRegistries::CheckUpstreamsService.new(
  registry: registry,
  params: { path: 'library/alpine/manifests/latest' }
)

result = service.execute
# => #<ServiceResponse:0x0000000324f15af0
# @http_status=:ok,
# @message=nil,
# @payload=
#  {:upstream=>
#    #<VirtualRegistries::Container::Upstream:0x00000003136d5518
#     id: 2,
#     group_id: 22,
#     created_at: Fri, 22 Aug 2025 08:18:57.958914000 UTC +00:00,
#     updated_at: Fri, 22 Aug 2025 09:15:26.684211000 UTC +00:00,
#     cache_validity_hours: 24,
#     username: "[FILTERED]",
#     password: "[FILTERED]",
#     url: "https://registry-1.docker.io",
#     name: "DockerHub Test",
#     description: "[FILTERED]">},
# @reason=nil,
# @status=:success>

result.success?
# => true

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #549104 (closed)

Edited by Adie (she/her)

Merge request reports

Loading