`agentk` should notify Flux about new commits without any extra configuration
Release notes
Problem to solve
As an Application Operator, I want changes committed for deployment to be picked up immediately by my preferred GitOps tool (Flux), so I don't have to wait for its regular reconciliation loop.
Today, Flux provides incoming webhook receivers that need to be set up to have Flux trigger a repository sync without waiting for its configured reconciliation loop. Setting up such a receiver is a non-trivial task as it requires a secret and configuration in the cluster and in GitLab.
Not in scope
Flux supports syncing from several repositories. In this iteration the agent would trigger a sync only for changes in the agent config repo.
Manifest projects outside of the Agent configur... (&7704 - closed) should provide support for multiple repositories with a single agent
Proposal
agentk:
Add a new module to agentk that:
-
runs an informer for Flux's
GitRepository
objects (see the remote development module that has similar functionality) with unstructured object using a dynamic client (don't depend on Flux go libraries). For each object it should ensure there is a correctly configuredReceiver
object (ageneric
receiver can be used in a first iteration with the option to switch to ageneric-hmac
later - the webhook endpoint to call is exposed instatus.webhookPath
which must be read by agentk), linked via owner references so that Kubernetes' GC cleans it up when theGitRepository
is deleted. -
(future iteration) It should run an informer for Flux's
Receiver
objects. When an object is deleted, mutated, it should trigger reconciliation of the owningGitRepository
object. This ensures that if someone is messing with our objects, we restore them to the correct state. We should also add some sort of label to allReceiver
objects we create so that we can ignore everything else. -
For each
GitRepository
object that wants to pull from a GitLab repo (we need to know the GitLab URL so that we can detect what is relevant and what is not) open a long-polling RPC to kas. It should sit and wait for notifications from kas. We should also see if it'd be easier (it'd be more efficient for sure) to just use a single RPC to subscribe to notifications for all the repos. Restart the RPC when list of repos changes. -
Once a notification is received from kas, the module should call the corresponding
Receiver
. -
All "writes" should use server-side apply.
Flux:
Depends on Hook into GitLab's internal/post_receive notifi... (gitlab-org/cluster-integration/gitlab-agent#43 - closed) - otherwise we don't get any events in KAS that new refs have been pushed.
Add a server module for Flux:
- Expose an endpoint to agentk to allow subscribing to "commit push" notifications.
- When an RPC is received, subscribe to notifications for listed repos via the mechanism, described in the link above.
- Unsubscribe when RPC ends.
Intended users
Feature Usage Metrics
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.