Add distributed tracing with LabKit
See labkit!2 (merged) for details
Gitaly Companion MR: gitaly!976 (merged)
GitLab-CE companion MRs: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21280 https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24186
Adds distributed tracing via LabKit.
Description
This change adds distributed tracing support through LabKit using OpenTracing APIs.
By default, no tracing implementation is linked into the Workhorse binary, but different OpenTracing providers can be linked in using build tags/build constraints. This can be done by setting the BUILD_TAGS
make variable.
For more details of the supported providers, see LabKit, but as an example, for Jaeger tracing support, include the tags: BUILD_TAGS="tracer_static tracer_static_jaeger"
.
make BUILD_TAGS="tracer_static tracer_static_jaeger"
Once Workhorse is compiled with an opentracing provider, the tracing configuration is configured via the GITLAB_TRACING
environment variable.
For example:
GITLAB_TRACING=opentracing://jaeger ./gitlab-workhorse
Testing
The best way to get a feel for how this works is to test it end-to-end. You can find detailed instructions of how to do this in the How to Test section here: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21280#how-to-test
Merge request reports
Activity
mentioned in merge request !295 (closed)
mentioned in merge request labkit!1 (merged)
added 53 commits
-
bc0a5fbb...1788824b - 52 commits from branch
master
- 509d507e - Merge branch 'master' of gitlab.com:gitlab-org/gitlab-workhorse into an-labkit-dtrace
-
bc0a5fbb...1788824b - 52 commits from branch
mentioned in merge request labkit!2 (merged)
mentioned in merge request gitaly!976 (merged)
added Distributed Tracing label
added Observability label
assigned to @jacobvosmaer-gitlab
changed milestone to %11.8
@jacobvosmaer-gitlab this is the same code that we vendored into Gitaly in gitaly!976 (merged), vendored into Workhorse.
Please could you review it?
gitlab-org/gitlab-ce!21280#how-to-test has details instructions of how to test end-to-end distributed tracing across the application.
Also, I would be happy to give you a demo of the entire work-piece in action if this would help you understand the benefits it brings.
- Resolved by Nick Thomas
assigned to @andrewn
assigned to @jacobvosmaer-gitlab
Assigning to @nick.thomas for second review just so that he knows about the new build tag options.
assigned to @nick.thomas
Thanks @jacobvosmaer-gitlab @andrewn
Mostly LGTM, but there are merge conflicts! Also,
vendor/vendor.json
hasn't changed? Surely it should?assigned to @andrewn
@nick.thomas Thank you! Investigating the
vendor.json
issue nowAlso related documentation MR, as requested by @jacobvosmaer-gitlab: labkit!7 (merged)
added 5 commits
-
90a24ada...254f37c7 - 4 commits from branch
master
- 0696e4c2 - Merge branch 'master' of gitlab.com:gitlab-org/gitlab-workhorse into an-labkit-dtrace
-
90a24ada...254f37c7 - 4 commits from branch
Also,
vendor/vendor.json
hasn't changed? Surely it should?@nick.thomas Strangely, this seems to be a UI bug of sorts in GitLab?
I can confirm that there are differences according to
git diff master..an-labkit-dtrace vendor/vendor.json
Also, they're visible in https://gitlab.com/gitlab-org/gitlab-workhorse/compare/master...an-labkit-dtrace (scroll to the bottom)
I've fixed the conflicts...
assigned to @nick.thomas
@andrewn that's weird :( the limit on diffs is 1000 files, and we're well under that.
Hah, when I look at
diffs.json
for this MR, I see Firefox tell me::SyntaxError: JSON.parse: unterminated string at line 1 column 1074725 of the JSON data
I'll dig into that more. In the meantime, happy to merge this
mentioned in commit c4f24a57