Should GitLab Kubernetes Agent Server become part of Workhorse?
GitLab Kubernetes Agent (the feature) is currently implemented as two components (architecture doc):
- GitLab Kubernetes Agent (
agentk
program). The component that user installs into their cluster - GitLab Kubernetes Agent Server (
kas
program). The server-side counterpart, that runs "next to GitLab".
The question is: should kas
be part of Workhorse or a standalone component or the GitLab deployment? Or maybe there is some other option?
Benefits of making it part of Workhorse:
-
Workhorse is already part of all the installation packages that we provide. We would not need to add a new piece to Omnibus, GitLab Chart, etc.
FWIW
kas
is already part of GDK and it was not hard to put it there. -
???
Challenges:
-
Depending on another team for reviews and merging code will likely slow down the development. Nobody on the Configure team is a Workhorse maintainer.
This can be mitigated by becoming maintainers and/or by reducing "integration surface". The later means structuring code in a way that minimizes changes to Workhorse during the development. In practice
kas
can be imported by Workhorse as a library and then changes are mostly limited to version bumps. See gitlab-workhorse!536 (closed). -
It may not be a good fit as
kas
will likely have some significant amount of business logic in it and mixing unrelated concerns in a single program is not great.From Workhorse readme:
Gitlab-workhorse is a smart reverse proxy for GitLab. It handles "large" HTTP requests such as file downloads, file uploads, Git push/pull and Git archive downloads.
Would be strange to also say that it's the GitLab<->Kubernetes integration server.
-
Logs, metrics, CPU and memory usage, crashes, memory leaks, etc of two programs become mixed together and hence monitoring and troubleshooting might be harder to perform. For on-call, which team is contacted when there is a problem?
-
???
Technical details
-
Merging connection pools will require some refactoring in Workhorse and in
kas
(trivial)Both Workhorse and
kas
need to talk to Gitaly and GitLab main application. In the futurekas
might also need to talk to Redis. For each of these systems some sort of connection pooling is in place. If integration surface between Workhorse andkas
is minimized (see above), then we'll have two connection pools for each remote system (i.e. two for Gitaly, two for GitLab and two for Redis) within a single program/process. This, of course, is a waste of resources (how big is the problem?). To fix that we'd need to do some refactoring in Workhorse andkas
to allow sharing each of those connection pools:-
Redis connection pool is a global, not an injected object.
-
Gitaly connection pool is a global, not an injected object.
-
GitLab connection pool is not a global and hence is likely easier to refactor for reuse.
Nothing is impossible, but globals might be a little bit harder to refactor to make reusable.
-
-
Integrating
kas
as a library will require to unify dependency versionsSee labkit!62 (closed) and gitlab-workhorse!536 (closed) for dependency version changes. Is that a concern? Sometimes updating versions is not desirable or possible for some non-obvious reasons, hence the question.
-
???
Please feel free to invite anyone who might be interested in this conversation.
@gitlab-org/configure @zj-gitlab @grzesiek @nick.thomas @nolith