Add application setting for CI telemetry OTEL Collector endpoint URL

Summary

Add a Rails application setting for the OTEL Collector endpoint URL used by CI Job Telemetry. Rails includes this URL in features.tracing.otel_endpoints when responding to runner job requests, so runners can push telemetry spans without static config.toml configuration.

Background

The architecture design document specifies that the OTEL Collector endpoint is sent from Rails in the job payload as features.tracing.otel_endpoints (array of objects, max 2). Each endpoint object carries a url and optional typed auth configuration (endpoint auth schema). This was decided to ensure the feature works automatically for any runner based on version and namespace plan, without requiring administrators to configure each runner manager individually (discussion).

Requirements

  1. Add an application setting (instance-level) for the primary OTEL Collector OTLP/HTTP endpoint URL
    • Setting name suggestion: ci_telemetry_otel_endpoint (string, nullable)
    • When blank/nil, features.tracing is not included in job payloads (telemetry disabled)
  2. Include the configured URL as the first endpoint object in features.tracing.otel_endpoints in the /api/v4/jobs/request response. The endpoint object includes the url and optional auth configuration (for GitLab.com hosted runners, this is http_bearer auth using GCE metadata identity tokens)
  3. The setting must be available for both GitLab.com (instance-level) and self-managed instances
  4. For GitLab.com, this will be configured by infrastructure to point to the Observability team's OTEL Collector

Future (not in scope for this issue)

  • Optional per-project/group BYO OTLP endpoint (second entry in otel_endpoints array, max 2 total) — tracked separately

Architecture Reference

Dependencies

  • Feature negotiation (#590588 (closed)) — features.tracing is only included when the feature flag is enabled and the endpoint is configured
  • Trace context initialization (#590587 (closed)) — needs to include otel_endpoints alongside trace_id
  • Sampling rate application setting (#593834 (closed)) — features.tracing is only included when the pipeline is also sampled

Parent Epic

&20945

Edited by Pedro Pombeiro