Show shared agents in deployment projects

Problem

Typically, agents are owned by a single project and shared with multiple deployment projects. Project members in the agent owner project can see the list of agents from "Infrastructure > Kubernetes clusters" page, however, project members in deployment projects might not have access to the owner project thus have no idea what agents are available in the project.

Similarly, we recently added a feature to allow users to access agent via user_access keyword. This authentication requires Agent ID, however, project members in the deployment projects might not have access to the owner project thus can't fetch the Agent ID for the authentication. This effectively limits the feature usage to be inside the owner project only.

Proposal

Show shared agents in deployment projects from "Infrastructure > Kubernetes clusters" page. In this page, users see:

• Shared agent names, IDs and it's connection status.
• Shared by - The agent owner project that allowed the access to the agent(s).
• Which keyword is used for sharing, either ci_access, user_access or both.
• Permissions in the cluster with the given impersonation. i.e. ClusterRoleBinding or RoleBinding details of the impersonated service account.
• Future iteration: View resources in the cluster via Kubernetes Dashboard. (user_access only)
• Future iteration: Connect to the Agent via Web Terminal. (user_access only)

Example design at the group-level here: #353468[Agent_Group_Installed_agents.png]

Permissions

Follow View clusters and Manage clusters from https://docs.gitlab.com/ee/user/permissions.html#project-members-permissions

Technical details

• One of main purposes in this iteration is to let deployment project see shared agents. To do so, we need to properly fix Agent permission in GitLab side.
• We likely use Clusters::Agents::ProjectAuthorization and Clusters::Agents::GroupAuthorization for fetching available agent list.
• Introduce read_cluster_agent permission. Stop reusing read_cluster as it technically infers deprecated certificate-based cluster integration.

Design

Reference

• How to share agents via ci_access => doc.
• How to share agents via user_access => doc.
• We show shared Deploy Keys in projects. See "Settings > Repository > Deploy Key" for the reference.

mentioned in merge request !104504 (merged)

@tkuah, Please add a group or category label to identify issue ownership.

You can refer to the Features by Group handbook page for guidance.

If you are unsure about the correct group, please do not leave the issue without a group label, and refer to GitLab's shared responsibility functionality guidelines for more information on how to triage this kind of issue.

This message was generated automatically. You're welcome to improve it.

added backend devopsconfigure [DEPRECATED] groupconfigure [DEPRECATED] sectionops labels

added to epic &8038

marked this issue as related to #389430 (closed)

changed title from User access for kas to Improve permission handling for user access with KAS

added ruby workflowrefinement labels

added featureenhancement label

added typefeature label

mentioned in issue gitlab-org/ci-cd/deploy-stage/environments-group/general#1 (closed)

@shinya.maeda This topic came up in a few places recently. Is this the best issue to track or is this a duplicate, WDYT?

marked this issue as related to #391610 (closed)

marked this issue as related to #353468 (closed)

changed title from Improve permission handling for user access with KAS to Show shared agents in deployment projects

changed the description

changed the description

changed the description

@nagyv-gitlab I've revised the proposal. This is actually related to #353468 (closed), but more specific to the problems we're currently facing.

cc @timofurrer @hfyngvason

@shinya.maeda Thanks. I linked to a (tangentially) related design from Ali.

@emilybauman I'll move this to workflowready for design as I'd like your input on the preferred columns. Unfortunately, I removed the proposal from Shinya and now we only have the group level designs from Ali (linked in the description).

Thanks @nagyv-gitlab - we can work to get this scheduled into an upcoming milestone for me.

@emilybauman For now, please, schedule it for %16.0

@nagyv-gitlab - Sounds good, I'll keep a note of this for 16.0

@nagyv-gitlab, @emilybauman, do we want to reflect on the UI if the agent belongs to a current project or is shared?

I've generated some ideas on how we can show it:

Idea	Screenshot
Add a separate tab
Add a column with a badge
Add a badge to the agent name

@anna_vovchenko - Thanks for putting together the proposals here!

I like option 3, but we could change the badges to be grey (info) and on the right side of the agent name. That being said, is there a good reason or data showing that users may want to have the agents separated out by project and shared? If so, we could look towards something like option 1.

@emilybauman, I was inspired by the runners' badges, but grey ones make sense. Here's an updated screenshot:

@anna_vovchenko - Now seeing it visualized I actually preferred your proposal. The thing that was throwing me off is green badges are supposed to be used when:

Success: Metadata that communicates success or completion with a positive meaning.

Let me do a bit of digging to see if green and blue are okay colour combos (or we could do blue and grey). I do see we do it in runners, so there might be some flexibility here.

@emilybauman, as an option, we could also not drive much user attention to the agent origin. We can add a small copy to the shared agent as clicking on its name will navigate the user to a different project:

I missed it in my previous screenshots, but we won't be able to show the agent's configuration for the shared agent as easily - we are only getting this configurations list per project.

@anna_vovchenko - I like that idea, we could even use just the dimmed grey badge here to call out shared agents and leave non-shared as is with no badge. It would show the difference quite well, but not overwhelm the page. Thanks for the proposals.

I missed it in my previous screenshots, but we won't be able to show the agent's configuration for the shared agent as easily - we are only getting this configurations list per project.

I think the idea of writing out external project or just some content to call this out should be enough. We can link them to the docs for MVC if anything.

Great, thanks, @emilybauman!

Here's a screenshot to summarize our discussion:

@emilybauman @anna_vovchenko LGTM. Thank you for the discussion!

That being said, is there a good reason or data showing that users may want to have the agents separated out by project and shared?

Nope. No need for tabs.

Looks great @anna_vovchenko! Thank you.

marked this issue as related to #396407 (closed)

mentioned in issue #396407 (closed)

changed epic to &9859 (closed)

removed the relation with #396407 (closed)

marked this issue as blocking #396407 (closed)

changed the description

added workflowready for design label and removed workflowrefinement label

changed the description

marked #391610 (closed) as a duplicate of this issue

mentioned in issue #391610 (closed)

mentioned in epic &9859 (closed)

changed the description

added groupenvironments label and removed groupconfigure [DEPRECATED] label

added devopsdeploy label and removed devopsconfigure [DEPRECATED] label

@anna_vovchenko Let me know if you need backend help I can work on it once the design is settled

Thanks, @shinya.maeda! I would appreciate your help! I think we've settled on the following design:

From the frontend side all that we need is some kind of a flag that the agent is shared in the clusterAgents GraphQL query.

@anna_vovchenko I started working on the extending the GraphQL query in Expose agent authorizations in GraphQL (!116064 - closed).

mentioned in issue gitlab-org/quality/triage-reports#11808 (closed)

I am currently looking at #396407 (closed) that is apparently blocked by this issue, as quoted here:

To do this, we likely need to finish Show shared agents in deployment projects (#395498 - closed) for letting deployment projects fetch shared agents. Currently, this is not possible due to lacking read_cluster permission.

There is also this line in this issue's description:

Introduce read_agent permission. Stop reusing read_cluster as it technically infers deprecated certificate-based cluster integration.

It sounds to me like both #396407 (closed) and this issue are blocked by the lack of the read_cluster permission in the deployment project?

Should we create an issue for that? It looks like #391610 (closed) is closed due to being duplicated, but the linked duplicate issue is #396407 (closed), which doesn't seem to be the same thing. I think we probably need to implement #396407 (closed) first then re-open #391610 (closed).

cc @shinya.maeda

To follow up on this, I think the order of implementation would be:

Task	Issue Link
1. Persist the GitLab agent's user_access permissions in the DB	#396407 (closed)
2. Introduce something like a read_cluster_agent policy that reads the persisted GitLab agent permissions (both from user_access and ci_access), and can be used from the deployment projects	#391610 (closed) (or we can open another one)
3. Show shared agents in deployment projects (this issue)	#395498 (closed)
4. Allow to Select Agent in Environment setting page	#396407 (closed)

And 3 and 4 can be implemented at the same time once 1 & 2 are done.

@partiaga Ah, that's good point. Yes, we need to finish Persist GitLab agent's user access configuratio... (#389430 - closed) as well, otherwise Clusters::AgentAuthorizationsFinder doesn't return user_access allowed agents.

At the moment, I think we don't need to do Allow users to read agent information if they'r... (#391610 - closed) as it's more like a hacky solution as compared to this issue. From data modeling perspective, we should have "shared agents" attributes/models inside the codebase, i.e. a deployment project fetch shared Clusters::Agent without explicitly querying to the agent management project.

Updated the Execution order

Thanks @shinya.maeda

Just to make sure I'm understanding this correctly:

1. Are we going to use the :read_cluster policy for checking access to these new features? (The :read_cluster is connected to the Project, so I think it should be okay.)
2. #396407 (closed) isn't blocked by anything related to policies, but by the ability to fetched the shared agents, which will be implemented in this issue.

Is my understanding here correct?

@partiaga

Are we going to use the :read_cluster policy for checking access to these new features? (The :read_cluster is connected to the Project, so I think it should be okay.)

For now, we're going to use :read_cluster policy, but we would create :read_cluster_agent if it's necessary. This allows us to perform more granular authorization with user.can?(:read_gitlab_agent_for_kubernetes, agent) (i.e. hook AgentPolicy), instead of checking it against user.can?(:read_cluster, agent.project) (i.e. ProjectPolicy). The output of DeclarativePolicy should look like following:

agent_management_project = Project.find()
agent_management_project_user = User.find()
shared_agent = agent_management_project.cluster_agents.first
agent_deployment_project = Project.find()
agent_deployment_project_user = User.find()

agent_management_project_user.can?(:read_cluster_agent, agent_management_project) # => true
agent_management_project_user.can?(:read_cluster_agent, shared_agent) # => true
agent_deployment_project_user.can?(:read_cluster_agent, agent_management_project) # => false, because it might contain other agents that are not shared.
agent_deployment_project_user.can?(:read_cluster_agent, shared_agent) # => true

#396407 (closed) isn't blocked by anything related to policies, but by the ability to fetched the shared agents, which will be implemented in this issue.

The ability to fetch shared agents should be guarded by policies. So I consider this issue is to do some groundwork that making sure fetch and policy are correctly implemented, so that we don't need to revisit the policy in Allow to Select Agent in Environment setting page (#396407 - closed) and it can simply use existing functionalities, such as finders.

assigned to @shinya.maeda

changed the description

mentioned in merge request !116064 (closed)

mentioned in merge request !116214 (merged)

mentioned in merge request !116368 (closed)