Update `jobs/allowed_agents` API to filter by environments

What does this MR do and why?

We need a way for users to restrict CI access to Kubernetes agents to selected environments. This can be done by adding environments in the groups and projects ci_access in the agent config saved in a project's repository.

Through kas, the ci_access.projects.environments OR ci_access.groups.environments configurations are cached/persisted in the database. (See MR: gitlab-org/cluster-integration/gitlab-agent!793 (merged))

In this MR, the GET /api/v4/job/allowed_agents API is updated to filter out the allowed agents for a job given the job's environment. The following conditions apply:

Agent has environments config?	Job has environment?	Job & Agent config matches?	Allowed?
No	No	N/A	Yes
No	Yes	N/A	Yes
Yes	Yes	Yes	Yes
Yes	No	No	No
Yes	Yes	No	No

Release/deployment considerations

This will not work end-to-end in production unless gitlab-org/cluster-integration/gitlab-agent!793 (merged) is deployed as part of the GitLab Agent's monthly release. Without that GitLab Agent change, the agent's environments configuration won't be persisted in the rails database, which means that the agents returned by the /job/allowed_agents API will not be filtered by environment regardless of the agentk configuration. This is the current behaviour, so there is no need to put this behind a feature flag while waiting for gitlab-org/cluster-integration/gitlab-agent!793 (merged) to be deployed.

Issue

#343885 (closed)

Screenshots or screen recordings

These tests match the conditions listed in the truth table above. The test job has no configured environment, while the deploy job is configured for production.

Agent Configuration	Job and status
Agent has no environment	Test job without environment - CAN ACCESS AGENT
Agent has no environment	Deploy job with production environment - CAN ACCESS AGENT
Agent has production environment	Deploy job with production environment - CAN ACCESS AGENT
Agent has production environment	Test job without environment - CANNOT ACCESS AGENT
Agent has dev environment	Deploy job in production environment - CANNOT ACCESS AGENT

How to set up and validate locally

1. Make sure your GitLab Agent Server (aka gitlab-k8s-agent, aka kas) is update to date with master and is enabled and running

2. Create the following projects in your local GitLab instance.

   • a project to setup the Kubernetes cluster connection, let's call this the agentk-setup
   • a project to test the CI/CD pipeline access to the Kubernetes cluster, let's call this the ci-k8s-access

3. In the agentk-setup project:

   1. Register an agent with GitLab
   2. Install the agent to a Kubernetes cluster. See instructions for Installing the agent in the cluster and Deploy the GitLab Agent (agentk) with k3d

4. Modify the environments access in the agent configuration file in the agentk-setup project to allow ci-k8s-access project to access the Kubernetes cluster:

   ci_access:
     projects:
       - id: ci-k8s-access
         environments:
           - dev
           - test
           - staging
           - review/*
           - production

   Here is an example of what it looks like on the Web IDE:

   

5. In the ci-k8s-access project, setup jobs through the CI pipeline editor. Test access to the Kubernetes agent installed in the previous steps from different jobs with different environments. E.g.:

   

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.