Multi-tenant pull-based syncing

Release notes

The GitLab Kubernetes Agent provides both push and pull based deployment possibilites for Kubernetes. Until now, pull-based deployments always happened using the same service account that the Agent was using in the cluster. This required separate agent instances by security groups, and resulted in a single agent per developer team. While the cluster side component has only minor resource requirements, having multiple agent instances running beside each other can become a sizeable requirement. The current version overcomes this problem, and allows platform engineers to set up a single agent to use different service accounts for the deployments depending on the source manifests.

Problem to solve

As a platform engineer, I want to run only a few agentk instances in my cluster to save on resources.

As a platform engineer, I want to manage RBAC per team, so that ownerships are clear within the cluster.

Proposal

Add impersonation features similar to what we have for ci_access.access_as. Maybe call it apply_as. For example:

gitops:
  manifest_projects:
    - id: my/project
      paths: [...]
      apply_as:
        # ONE of the below:
        agent: {} # current behavior
        manifest_project: {} # impersonate the manifest project
        impersonate: # fully custom impersonation
          username: some-user
          groups: [...]

From @hfyngvason:

I think it should be relatively simple to implement by following the same path as for ci_access.access_as:

• apply_as.impersonate and apply_as.agent: Can be implemented without any changes to GitLab Rails
• apply_as.manifest_project:
  • Decide what the user and groups should be called. If we are not sure, we can leave apply_as.manifest_project for a future iteration.
  • Add any missing metadata to the rails auth endpoint
  • Update the code over here to make use of the metadata

Questions

What would be the right approach?

• We can have a single agent configuration project that contains many gitops directories to sync, and each directory specifies the service account used
  • Here a big limitation is that each team is dependent on the platform team who owns the configuration project
• A single agent is configured through multiple projects somehow, and the central configuration project specifies only the service account used by each sub-configuration
  • How can we support this approach? What about an include syntax and codeowners could regulate how can change each template-part.