Ship GitLab with built-in Jaeger support for Distributed Tracing
GitLab is incrementally having distributed tracing support added to the application.
- GitLab-Workhorse, Gitaly, Sidekiq and Rails are instrumented with opentracing instrumentation
- Sidekiq and Rails ship with the Jaeger opentracing instrumentation and client built-in (via Ruby gems)
- GitLab-Workhorse and Gitaly include support for Jaeger opentracing instrumentation (as well as Lightstep and Datadog) but this is not included in the binary at compile time.
- GitLab Developer Kit enables Jaeger at compile-time for Gitaly and Workhorse for all GDK users.
In light of this, I propose we take the next step and compile Workhorse and Gitaly with Jaeger compiled into the application.
Note that, by default, this will remain switched off. The GITLAB_TRACING
environment variable needs to be configured before services are instrumented.
Compiling this support into Gitaly and Workhorse gives us the option, particularly in production and support incidents, to use the distributed tracing functionality to diagnose issues.
Without this change, Workhorse and Gitaly will need to be recompiled onsite to enable Jaeger support.
This adds about 1mb to the size of the binary, but in my testing did not affect the memory footprint unless tracing was actually enabled (which, by default it is not)
This change is implemented in two MRS: