Allow agent CI access sharing across top level groups

Tiger Watsonrequested to merge
474225-add-cross-top-level-agent-ci-sharing into master

What does this MR do and why?

Allow agent CI access sharing across top level groups

Previously a project/group that was authorized to use the CI tunnel had to share a root ancestor with the agent configuration project.

On GitLab.com this is typically not a problem, as most projects/groups that would be authorized already belong to the same top level group. However, self-managed installations often structure their hierarchies differently with multiple top level groups, which limits the usefulness of the agent.

With this change, any project or group on the instance can be specified in the agent configuration file, regardless of root ancestor.

This functionality will not be available on GitLab.com, and is controlled by the organization_cluster_agent_authorization_enabled application setting.

References

Query changes

The complexity of the queries is reduced slightly due to one less where condition:

Project level

before 
SELECT 
  "agent_project_authorizations".* 
FROM 
  "agent_project_authorizations" 
  INNER JOIN "cluster_agents" ON "cluster_agents"."id" = "agent_project_authorizations"."agent_id" 
  INNER JOIN "projects" ON "projects"."id" = "cluster_agents"."project_id" 
WHERE 
  "agent_project_authorizations"."project_id" = 56215512 
  AND (
    config -> 'access_as' IS NULL 
    OR config -> 'access_as' = '{}' 
    OR config -> 'access_as' ? | array[ 'agent', 
    'ci_job', 
    'ci_user', 
    'impersonate' ]
  ) 
  AND "projects"."namespace_id" IN (
    SELECT 
      "namespaces"."id" 
    FROM 
      "namespaces" 
    WHERE 
      "namespaces"."type" = 'Group' 
      AND (
        traversal_ids @> ('{9970}')
      )
  )
after 
SELECT 
  "agent_project_authorizations".* 
FROM 
  "agent_project_authorizations" 
  INNER JOIN "cluster_agents" ON "cluster_agents"."id" = "agent_project_authorizations"."agent_id" 
  INNER JOIN "projects" ON "projects"."id" = "cluster_agents"."project_id" 
WHERE 
  "agent_project_authorizations"."project_id" = 56215512 
  AND (
    config -> 'access_as' IS NULL 
    OR config -> 'access_as' = '{}' 
    OR config -> 'access_as' ? | array[ 'agent', 
    'ci_job', 
    'ci_user', 
    'impersonate' ]
  )

Group level

before 
WITH "ordered_ancestors" AS MATERIALIZED (
  SELECT 
    "namespaces"."id" 
  FROM 
    (
      SELECT 
        "namespaces"."id", 
        "namespaces"."name", 
        "namespaces"."path", 
        "namespaces"."owner_id", 
        "namespaces"."created_at", 
        "namespaces"."updated_at", 
        "namespaces"."type", 
        "namespaces"."description", 
        "namespaces"."avatar", 
        "namespaces"."membership_lock", 
        "namespaces"."share_with_group_lock", 
        "namespaces"."visibility_level", 
        "namespaces"."request_access_enabled", 
        "namespaces"."ldap_sync_status", 
        "namespaces"."ldap_sync_error", 
        "namespaces"."ldap_sync_last_update_at", 
        "namespaces"."ldap_sync_last_successful_update_at", 
        "namespaces"."ldap_sync_last_sync_at", 
        "namespaces"."description_html", 
        "namespaces"."lfs_enabled", 
        "namespaces"."parent_id", 
        "namespaces"."shared_runners_minutes_limit", 
        "namespaces"."repository_size_limit", 
        "namespaces"."require_two_factor_authentication", 
        "namespaces"."two_factor_grace_period", 
        "namespaces"."cached_markdown_version", 
        "namespaces"."project_creation_level", 
        "namespaces"."runners_token", 
        "namespaces"."file_template_project_id", 
        "namespaces"."saml_discovery_token", 
        "namespaces"."runners_token_encrypted", 
        "namespaces"."custom_project_templates_group_id", 
        "namespaces"."auto_devops_enabled", 
        "namespaces"."extra_shared_runners_minutes_limit", 
        "namespaces"."last_ci_minutes_notification_at", 
        "namespaces"."last_ci_minutes_usage_notification_level", 
        "namespaces"."subgroup_creation_level", 
        "namespaces"."max_pages_size", 
        "namespaces"."max_artifacts_size", 
        "namespaces"."mentions_disabled", 
        "namespaces"."default_branch_protection", 
        "namespaces"."max_personal_access_token_lifetime", 
        "namespaces"."push_rule_id", 
        "namespaces"."shared_runners_enabled", 
        "namespaces"."allow_descendants_override_disabled_shared_runners", 
        "namespaces"."traversal_ids", 
        "namespaces"."organization_id", 
        ABS(
          2 - array_length(traversal_ids, 1)
        ) as depth 
      FROM 
        "namespaces" 
      WHERE 
        "namespaces"."type" = 'Group' 
        AND "namespaces"."id" IN (9970, 2287432, 64689037, 72590210)
    ) namespaces 
  WHERE 
    "namespaces"."type" = 'Group' 
  ORDER BY 
    "depth" ASC
) 
SELECT 
  DISTINCT ON (agent_id) agent_group_authorizations.* 
FROM 
  "agent_group_authorizations" 
  INNER JOIN "ordered_ancestors" ON "agent_group_authorizations"."group_id" = "ordered_ancestors"."id" 
  INNER JOIN "cluster_agents" ON "cluster_agents"."id" = "agent_group_authorizations"."agent_id" 
  INNER JOIN "projects" ON "projects"."id" = "cluster_agents"."project_id" 
WHERE 
  (
    config -> 'access_as' IS NULL 
    OR config -> 'access_as' = '{}' 
    OR config -> 'access_as' ? | array[ 'agent', 
    'ci_job', 
    'ci_user', 
    'impersonate' ]
  ) 
  AND "projects"."namespace_id" IN (
    SELECT 
      "namespaces"."id" 
    FROM 
      "namespaces" 
    WHERE 
      "namespaces"."type" = 'Group' 
      AND (
        traversal_ids @ > ('{9970}')
      )
  ) 
ORDER BY 
  agent_id, 
  array_position(
    ARRAY(
      SELECT 
        id 
      FROM 
        ordered_ancestors
    ):: bigint[], 
    agent_group_authorizations.group_id
  )
after 
WITH "ordered_ancestors" AS MATERIALIZED (
  SELECT 
    "namespaces"."id" 
  FROM 
    (
      SELECT 
        "namespaces"."id", 
        "namespaces"."name", 
        "namespaces"."path", 
        "namespaces"."owner_id", 
        "namespaces"."created_at", 
        "namespaces"."updated_at", 
        "namespaces"."type", 
        "namespaces"."description", 
        "namespaces"."avatar", 
        "namespaces"."membership_lock", 
        "namespaces"."share_with_group_lock", 
        "namespaces"."visibility_level", 
        "namespaces"."request_access_enabled", 
        "namespaces"."ldap_sync_status", 
        "namespaces"."ldap_sync_error", 
        "namespaces"."ldap_sync_last_update_at", 
        "namespaces"."ldap_sync_last_successful_update_at", 
        "namespaces"."ldap_sync_last_sync_at", 
        "namespaces"."description_html", 
        "namespaces"."lfs_enabled", 
        "namespaces"."parent_id", 
        "namespaces"."shared_runners_minutes_limit", 
        "namespaces"."repository_size_limit", 
        "namespaces"."require_two_factor_authentication", 
        "namespaces"."two_factor_grace_period", 
        "namespaces"."cached_markdown_version", 
        "namespaces"."project_creation_level", 
        "namespaces"."runners_token", 
        "namespaces"."file_template_project_id", 
        "namespaces"."saml_discovery_token", 
        "namespaces"."runners_token_encrypted", 
        "namespaces"."custom_project_templates_group_id", 
        "namespaces"."auto_devops_enabled", 
        "namespaces"."extra_shared_runners_minutes_limit", 
        "namespaces"."last_ci_minutes_notification_at", 
        "namespaces"."last_ci_minutes_usage_notification_level", 
        "namespaces"."subgroup_creation_level", 
        "namespaces"."max_pages_size", 
        "namespaces"."max_artifacts_size", 
        "namespaces"."mentions_disabled", 
        "namespaces"."default_branch_protection", 
        "namespaces"."max_personal_access_token_lifetime", 
        "namespaces"."push_rule_id", 
        "namespaces"."shared_runners_enabled", 
        "namespaces"."allow_descendants_override_disabled_shared_runners", 
        "namespaces"."traversal_ids", 
        "namespaces"."organization_id", 
        ABS(
          2 - array_length(traversal_ids, 1)
        ) as depth 
      FROM 
        "namespaces" 
      WHERE 
        "namespaces"."type" = 'Group' 
        AND "namespaces"."id" IN (9970, 2287432, 64689037, 72590210)
    ) namespaces 
  WHERE 
    "namespaces"."type" = 'Group' 
  ORDER BY 
    "depth" ASC
) 
SELECT 
  DISTINCT ON (agent_id) agent_group_authorizations.* 
FROM 
  "agent_group_authorizations" 
  INNER JOIN "ordered_ancestors" ON "agent_group_authorizations"."group_id" = "ordered_ancestors"."id" 
  INNER JOIN "cluster_agents" ON "cluster_agents"."id" = "agent_group_authorizations"."agent_id" 
  INNER JOIN "projects" ON "projects"."id" = "cluster_agents"."project_id" 
WHERE 
  (
    config -> 'access_as' IS NULL 
    OR config -> 'access_as' = '{}' 
    OR config -> 'access_as' ? | array[ 'agent', 
    'ci_job', 
    'ci_user', 
    'impersonate' ]
  ) 
ORDER BY 
  agent_id, 
  array_position(
    ARRAY(
      SELECT 
        id 
      FROM 
        ordered_ancestors
    ):: bigint[], 
    agent_group_authorizations.group_id
  )

How to set up and validate locally

  1. Go to Admin -> Settings -> General -> GitLab agent for Kubernetes, and enable Enable instance level authorization.
  2. Create a new top-level group, and create a project within it to be the agent configuration project. For example, group-1/agent-project.
  3. Create a second top-level group, and create a project to be the authorised project. For example, group-1/authorised-project.
  4. Register an agent in the agent configuration project using the following config: 
     ci_access:
   groups:
     - id: group-2 # or whatever you named the second group
  5. In the authorised project, add a .gitlab-ci.yml with the following content: 
     deploy-agent:
   image:
     name: bitnami/kubectl
     entrypoint: ['']
   script:
     - kubectl config get-contexts
  6. In the resulting CI job output, check there is a context for your agent in the format path/to/project:agent-name.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Fabien Catteau

Merge request reports