Add the policy store list endpoint

Marcos Rocharequested to merge
mc_rocha-policy-store-list-endpoint into master

What does this MR do and why?

Adds GET /organizations/:id/security/policy_store, the first Policy Store route that returns policy records rather than a static catalogue. It resolves the organization with find_organization!, checks read_govern_policy on it, and presents ListService's payload through the new API::Entities::Govern::Policy.

The organization tenancy this route needs was split out into !249226 (merged), which has merged. There are no service changes here, and the store gem is untouched.

The policy entity carries the gem's current field set, after the schema alignment landed on master: trigger_type rather than trigger_id, plus namespace_id, description, created_at, and updated_at.

Design decisions

read_govern_policy is enabled for organization owners and instance admins, on EE::Organizations::OrganizationPolicy, matching the cd_application and create_duo_flow_callback_hook blocks already in that file. Not access_organization_admin_area, which means "can open the organization admin area" and is gated behind the org_admin_area release, so reusing it would tie policy authoring to an unrelated rollout.

The granular token boundary is instance. Authz::Validation::BOUNDARIES is %w[instance group project user], so there is no organization boundary to declare. Every organization-scoped route in lib/api/organizations.rb does the same, including DELETE /organizations/:id. The consequence is that a granular token scoped to a group cannot call this endpoint. Two examples in the granular token shared example report as pending for the same reason: they only assert on project and group boundaries.

Unlike the catalogue routes, this one authenticates. A caller who is not an organization owner gets 403 for a public organization and 404 for a private one, since find_organization! resolves nothing it cannot read.

The experiment gate is checked twice, deliberately. The shared before block checks the feature flag, the instance setting, and the licence, and Organizations::Organization#policy_store_experiment_active? checks the same three, so ListService cannot report the experiment inactive to a request that got past the gate. The service keeps gating itself for callers that do not come through the API, which is why the request spec stubs ListService to cover that branch.

doc/api/openapi/openapi_v3.yaml does not change. The route is hidden true, so it contributes only the policy_store tag, and that tag already exists.

How to set up and validate locally

Requires an Ultimate licence.

Gitlab::PolicyStore is backed by an in-memory repository built lazily per process, and nothing configures it at boot, so a policy created on the rails console is invisible to the process serving the request. Seeding therefore happens in an initializer, which runs in every process including Puma. The ActiveRecord-backed repository in https://gitlab.com/gitlab-org/gitlab/-/work_items/606969 removes the need for this.

  1. Enable the security_policies_v2 feature flag on the rails console
Feature.enable(:security_policies_v2)
  1. As an administrator, go to Admin > Settings > Security and compliance and turn on the policy store experiment
  2. On the rails console, confirm the gate is satisfied, pick an organization you own, and note your GDK's base URL
organization = Organizations::Organization.find(1)
organization.policy_store_experiment_active?   # => true

puts Gitlab.config.gitlab.url   # for example https://gdk.test:3443
  1. Make yourself an owner of that organization, if you are not already
user = User.find_by_username('root')
Organizations::OrganizationUser.find_or_create_by!(organization: organization, user: user) do |organization_user|
  organization_user.access_level = Gitlab::Access::OWNER
end
  1. Seed a policy into the process that serves the request, by adding config/initializers/zz_temporary_policy_store_seed.rb, then running gdk restart rails-web. Delete the file when you are done
# frozen_string_literal: true

Rails.application.config.after_initialize do
  next unless Rails.env.development?

  Gitlab::PolicyStore.create(
    organization_id: 1,
    name: 'Block deployments on critical findings',
    trigger_type: 'deployment_requested'
  )
end
  1. Create a personal access token for yourself: select your avatar in the upper-right corner, select Edit profile, then in the left sidebar select Access > Personal access tokens
  2. From the Generate token dropdown list select Legacy token, enter a Token name, leave Expiration date empty, select the api scope, then select Generate token. Copy the token, since it is shown only once
  3. List the policies with that token, against the base URL from step 3
curl --header "PRIVATE-TOKEN: <your token>" \
  --url "<your GDK URL>/api/v4/organizations/1/security/policy_store"
  1. Verify the response carries the seeded policy, with organization_id matching the organization from step 3
[
  {
    "id": 1,
    "organization_id": 1,
    "namespace_id": null,
    "name": "Block deployments on critical findings",
    "description": null,
    "version": 1,
    "trigger_type": "deployment_requested",
    "rules": [],
    "actions": [],
    "policy_scope": null,
    "scope_rego": "package gitlab.scope\n\nimport rego.v1\n...",
    "mode": "enforce",
    "lifecycle_state": "active",
    "created_at": null,
    "updated_at": null
  }
]

namespace_id is null because the policy is owned by the organization rather than by a group, and scope_rego holds the applies-to-all program the store compiles when a policy has no scope. created_at and updated_at stay null until the ActiveRecord-backed repository replaces the in-memory one.

References

Edited by Marcos Rocha

Merge request reports