Loading
Add Prometheus instrumentation to Rails-side Orbit components
What does this MR do and why?
Adds Prometheus histograms and counters to the Rails-side Orbit components (GrpcClient, JwtAuth, AuthorizationContext, RedactionService). The GKG/Rust side has query pipeline timing and ClickHouse profiling, but the Rails hot path had no metrics, so we couldn't tell where latency was coming from.
This is the instrumentation MR in the redaction scalability series. See orbit/knowledge-graph#348 for the full plan.
New metrics module ee/lib/gitlab/metrics/knowledge_graph/request.rb, following the existing TraversalIds pattern:
| Metric | Component | Description |
|---|---|---|
gitlab_knowledge_graph_grpc_duration_seconds |
GrpcClient | Duration per RPC method (method/status labels) |
gitlab_knowledge_graph_grpc_errors_total |
GrpcClient | Errors by method and gRPC status code |
gitlab_knowledge_graph_jwt_build_duration_seconds |
JwtAuth | End-to-end generate_token duration (includes auth context) |
gitlab_knowledge_graph_auth_context_duration_seconds |
AuthorizationContext | Traversal ID computation on cache miss |
gitlab_knowledge_graph_redaction_duration_seconds |
RedactionService | Duration of execute (resource loading + ability checks) |
gitlab_knowledge_graph_redaction_batch_size |
RedactionService | Resources evaluated per batch |
gitlab_knowledge_graph_redaction_filtered_count |
RedactionService | Resources denied per batch |
Design decisions
measure_grpcusesensureso duration is always recorded, even on exception (status label becomes"error")RedactionServiceacceptsmetrics_observer:— a callable dependency injected by the caller, matching the existinglogger:pattern. This keeps the FOSS service generic and decouples it from EE metrics constants. The Workhorse internal redaction endpoint can inject its own observer.redaction_duration_secondsmeasuresRedactionService.execute— timed inside the service itself (resource loading + ability checks), not the gRPC streaming exchange. This survives the Workhorse migration (ADR 008) sinceRedactionServiceis called from both Rails and the Workhorse internal redaction endpoint.- FOSS compatibility — metrics calls live in EE code only; the FOSS
RedactionServiceuses the injected observer without referencing any EE constants.
How to set up and validate locally
git checkout arivera/rails-metrics-capture
gdk start
bundle exec rails runner "Feature.enable(:knowledge_graph)"Hit any orbit endpoint:
curl -sk --header "PRIVATE-TOKEN: $TOKEN" "https://gdk.test:3443/api/v4/orbit/schema"Then check /-/metrics:
curl -sk https://gdk.test:3443/-/metrics | grep gitlab_knowledge_graph_Verified output
gitlab_knowledge_graph_grpc_duration_seconds_sum{method="get_graph_schema",status="ok"} 0.006817
gitlab_knowledge_graph_grpc_duration_seconds_count{method="execute_query",status="error"} 1
gitlab_knowledge_graph_jwt_build_duration_seconds_sum 0.099997
gitlab_knowledge_graph_auth_context_duration_seconds_sum 0.096225
gitlab_knowledge_graph_redaction_duration_seconds_sum 0.198219
gitlab_knowledge_graph_redaction_batch_size_sum 1Edited by Michael Angelo Rivera