Draft: LabKit v2: a shared foundation for Go services at GitLab

Overview

This MR introduces LabKit v2 — a ground-up redesign of the shared Go library used across GitLab's Go services. Where LabKit v1 was a collection of loosely-related utilities, v2 is a coherent developer platform: a set of packages that compose together through a common application lifecycle, a unified observability stack, and consistent constructor patterns.

The goal is to cut the time it takes to bootstrap a production-ready GitLab Go service from days to minutes, while ensuring every service inherits the same logging format, tracing semantics, health check endpoints, and secret handling without any per-service configuration decisions.

What's included

app — Application lifecycle

The central coordinator. Services register pluggable Component implementations (database pool, HTTP server, etc.) and App handles ordered startup and reverse-ordered graceful shutdown.

a, _ := app.New(ctx)

db, _ := postgres.NewWithConfig(&postgres.Config{DSN: dsn, Tracer: a.Tracer()})
srv := httpserver.NewWithConfig(&httpserver.Config{Logger: a.Logger(), Tracer: a.Tracer()})

a.Register(db)
a.Register(srv)

a.Start(ctx)      // starts db, then srv
a.Shutdown(ctx)   // shuts down srv, then db (reverse order)

Any struct that implements Start(ctx) error and Shutdown(ctx) error can be registered. This makes it easy to add new infrastructure components without touching startup wiring.

httpserver — HTTP server with built-in observability

A production-ready HTTP server that implements app.Component and ships with liveness/readiness probes out of the box.

srv := httpserver.NewWithConfig(&httpserver.Config{
    Addr:            ":8080",
    ShutdownTimeout: 30 * time.Second,
    Logger:          logger,
    Tracer:          tracer,
})

srv.AddReadinessCheck("database", db.Pool().Ping)

srv.Router().HandleFunc("/api/v1/projects", handleProjects)

/-/liveness and /-/readiness are registered automatically. Readiness checks are chainable and run concurrently under the hood.

httpclient — Outbound HTTP with tracing, logging, and retry

A drop-in http.Client replacement that automatically propagates trace context and logs outbound requests. DoWithRetry respects Retry-After headers and is configurable per call.

client := httpclient.NewWithConfig(&httpclient.Config{
    Tracer: tracer,
    Logger: logger,
})

resp, err := client.DoWithRetry(req, &httpclient.RetryConfig{
    MaxAttempts:     3,
    RetryableStatus: []int{429, 503},
    MaxDelay:        60 * time.Second,
})

postgres — Instrumented PostgreSQL

A pgxpool-backed connection pool that implements app.Component and automatically traces every query with OpenTelemetry spans.

db, _ := postgres.NewWithConfig(&postgres.Config{
    DSN:      "postgres://user:pass@localhost/mydb",
    MaxConns: 20,
    Tracer:   tracer,
})
// db.Start(ctx) verifies connectivity and appears in the trace
pool := db.Pool() // *pgxpool.Pool — full pgx API available

featureflag — OpenFeature client with tracing

Evaluates boolean and string feature flags via GO Feature Flag relay proxy. Each evaluation is automatically recorded as a child span with the flag key, resolved value, variant, and reason.

ff, _ := featureflag.NewWithConfig(ctx, app, &featureflag.Config{
    Endpoint: "https://ff-relay.internal",
})

enabled, _ := ff.BoolFlag(ctx, "new-pipeline-engine", userID, false)

secret — Swappable secret provider

A thin abstraction over secret backends with a Secret type that redacts itself in logs and stack traces. Providers are swappable: EnvProvider for local dev, FileProvider for Kubernetes-mounted secrets.

client := secret.NewClient(secret.NewFileProvider("/var/run/secrets"))
apiKey, _ := client.Get(ctx, "third-party-api-key")

log.Info("fetching data", slog.String("key", apiKey.String())) // logs "[REDACTED]"
apiKey.Value() // actual value, used only at call site

trace — OpenTelemetry distributed tracing

A thin wrapper over the OTEL Go SDK. Configurable via GITLAB_TRACING environment variable in production or Config for programmatic control. Embeds oteltrace.Span to expose the full OTEL API.

tracer, shutdown, _ := trace.NewWithConfig(ctx, &trace.Config{
    ServiceName: "gitaly",
    SampleRate:  0.1,
})
defer shutdown(ctx)

ctx, span := tracer.Start(ctx, "resolve-ref")
defer span.End()

span.SetAttribute("ref", ref)
span.RecordError(err) // sets status=Error and records exception event

log — Structured logging with canonical field helpers

A slog-backed JSON logger. Field helpers return typed slog.Attr values keyed to GitLab's structured logging standard, eliminating typos in field names across services.

logger.InfoContext(ctx, "request complete",
    log.CorrelationID(correlationID),
    log.HTTPStatusCode(resp.StatusCode),
    log.DurationS(time.Since(start)),
    log.GitLabUserID(userID),
)

Configurable at runtime via GITLAB_LOG_LEVEL and GITLAB_LOG_FORMAT environment variables — no code changes needed for production log level adjustments.

Design principles

• Compose, don't configure: app.App threads the logger and tracer through to every subsystem automatically, so there's no per-service wiring boilerplate.
• Nil-safe by default: Span methods (SetAttribute, RecordError, End) are all nil-safe, so instrumentation code can be written without defensive nil checks.
• Environment-first configuration: Logging level/format and tracing endpoint are all env-var driven — compatible with GitLab's Kubernetes deployment model out of the box.
• Standard constructors: Every package provides New() (sensible defaults) and NewWithConfig(cfg) (explicit control). The same pattern everywhere means engineers can move between packages without relearning the API.
• Testability built in: app.NewForTesting, tracetest.NewRecorder, and logtest.NewWithRecorder provide in-process test doubles for every observable subsystem.