Match Flux push events on a GitLab with a relative URL root
Closes #1240 (closed)
On a GitLab served under a relative URL root, agentk registers a GitRepository cloned over HTTP under the path of its URL, root included (gitlab/some-org/some-repo), while push events carry the project's full path (some-org/some-repo). kas matched the two exactly (server.go#L83-85), so it never notified the agent, and Flux reconciled only on its interval.
kas now also matches a registered path with the root removed, and sends the agent each path it registered for the pushed project, because agentk looks Receivers up by the string it sent. Each registered path still matches as is: an SSH URL's path carries no root, and a top-level group can have the root's name. The project access check still uses the full path from the push event.
kas takes the root from the path of gitlab.address, so this needs no new setting and no change in Omnibus or the Helm chart. In the issue I asked you to pick an option first and said I would not rely on that field; reading Workhorse and Rails changed my mind, and since this needs no new setting I am sending the code now. I will rework it if you prefer another option. Workhorse answers 404 to any path outside its relative URL root (upstream.go#L197-203), and Rails is mounted under RAILS_RELATIVE_URL_ROOT (config.ru#L12). Since !659 (merged), kas keeps the path of gitlab.address when it builds API URLs (tools.go#L159-161), so it reaches the API only when that address carries the root, and both packaged deployments set it that way: Omnibus defaults gitlab_kas['gitlab_address'] to external_url (gitlab_kas.rb#L35), and the chart sets gitlab.address to the Workhorse URL with relativeUrlRoot appended (_default-config.yaml#L19, _workhorse.tpl#L18-20). agentk does not change, so the agents already deployed are notified as soon as kas is upgraded.
The other options from the issue, and why I did not take them:
- agentk strips the path of
gitlab.external_url. Omnibus drops the root from that field, so this would also need Omnibus, the field's documented format and theOrigincheck of the Kubernetes API proxy to change, and it would only help upgraded agents. - A new kascfg setting for the root, as the issue proposed it. It would repeat what
gitlab.addressalready carries, and nothing would change until each deployment set it. If you would rather have the root stated explicitly, I can add it as an optional override of the derived value. - kas passes the root to agentk in the agent configuration. That is a new field and an agentk change, so again only upgraded agents would be notified, and kas would still have to learn the root first.
A few notes for the review:
- Each registered path also matching as is has a cost: when a top-level group has the root's name, or when
gitlab.addresscarries a path that is not the root (kas behind a proxy that rewrites paths), a push can notify aReceiverthat did not need it, and Flux reconciles once for nothing. No notification that is sent today stops being sent. - When kas reaches GitLab through a proxy that adds the root,
gitlab.addresscarries no path and nothing changes from today; the optional setting in point 2 would cover that. - A failed
Sendstops sending the remaining paths for that push, since the stream is gone. The loop then goes on as before. TestReconcileProjects_RelativeURLRootdrivesReconcileProjectswithgitlab/some-org/some-reporegistered and a push forsome-org/some-repo, and checks that the agent is sentgitlab/some-org/some-repoand that the access check asks only about the paths GitLab pushed.TestProjectsByFullPathcovers the mapping (HTTP and SSH URLs, a root of two segments, a group whose name starts like the root) andTestGitLabRelativeURLRootthe address.
Tests on master and on this branch
On master 3b5c32b25 with only TestReconcileProjects_RelativeURLRoot added, minus the gitLabRelativeURLRoot field master does not have, the agent is sent only the last notification:
$ go test -race -count=1 -run 'TestReconcileProjects' ./internal/module/flux/server/
--- FAIL: TestReconcileProjects_RelativeURLRoot (0.00s)
expected: []string{"gitlab/some-org/some-repo", "other-org/other-repo"}
actual : []string{"other-org/other-repo"}
FAILOn this branch:
$ go test -race -count=1 -v -run 'TestReconcileProjects|TestProjectsByFullPath|TestGitLabRelativeURLRoot' ./internal/module/flux/server/
--- PASS: TestGitLabRelativeURLRoot (0.00s)
--- PASS: TestProjectsByFullPath (0.00s)
--- PASS: TestReconcileProjects_RelativeURLRoot (0.00s)
ok gitlab.com/gitlab-org/cluster-integration/gitlab-agent/v19/internal/module/flux/server
$ go test -race -count=200 -run 'TestReconcileProjects_RelativeURLRoot' ./internal/module/flux/server/
ok gitlab.com/gitlab-org/cluster-integration/gitlab-agent/v19/internal/module/flux/server
$ go test -race -count=1 ./internal/module/flux/...
ok gitlab.com/gitlab-org/cluster-integration/gitlab-agent/v19/internal/module/flux/agentk
ok gitlab.com/gitlab-org/cluster-integration/gitlab-agent/v19/internal/module/flux/server
$ go build ./... && go vet ./internal/module/flux/... ./internal/cmd/kas/...
$ golangci-lint fmt --diff ./internal/module/flux/...
$ golangci-lint run ./internal/module/flux/...
$ ./build/deadcode.sh v0.52.0
$ go run gitlab.com/gitlab-org/labkit/v2/cmd/validate-log-fields@v2.65.0 .
No deprecated logging field usages found.golangci-lint is the custom build from .custom-gcl.yml at v2.15.0; Go 1.27.3, linux/amd64. I did not run Bazel. A standalone gazelle (v1.0.0-beta.1, -mode=diff) reports no change to BUILD.bazel, and the same run on master with the new test lists exactly the dependencies this MR adds.