Add Rego scope transpiler to Policy Store gem

Marcos Rocharequested to merge
mc_rocha-add-rego-scope-transpiler into master

What does this MR do and why?

Compiles a policy's policy_scope (jsonb) into scope_rego text (package gitlab.scope) on create, through a new Gitlab::PolicyStore::ScopeTranspiler in the gitlab-policy-store gem.

What is in scope: converting scopes that are already valid. The input contract is ee/app/validators/json_schemas/security_policy_scope.json, the same schema existing policies validate against, so every scope that can be authored today compiles. Rejecting invalid scopes stays with that validator, and validating user-authored Rego is not part of this MR.

Scope reaches a policy one of two ways, and scope_rego is always the form that gets evaluated:

  • Structured data in policy_scope: compiled into scope_rego on create.
  • Rego supplied directly in scope_rego: stored as authored, without validation, and policy_scope is cleared so the two cannot disagree.

A policy with neither compiles to a program that applies everywhere, so scope_rego is never blank after create.

Matching the current scope semantics

security_policy_scope.json requires id only for security attributes, so [{}] is a valid entry under compliance_frameworks, projects, and groups. PolicyScopeChecker reads such an entry as a condition that is defined but that no project satisfies, so the policy applies to nothing. The transpiler draws the same line, carrying whether a criterion was declared alongside the ids that survived parsing, and emitting Rego's empty set literal (framework_id in set()) when a criterion was declared but names nothing. Collapsing the two would turn a policy that applies to no project into one that applies to every project. Six examples pin these scopes to the answers the checker gives for them.

Past that point the two are free to diverge: v2 policies live in their own table with no migration from the existing ones, so a dimension need not exist on both sides, and nothing asserts that they stay aligned.

What changed outside the transpiler

  • with_compiled_scope, one private helper on the port holding both the compile and the clear-policy_scope rules, called from the in-memory adapter's create after validation. A future ActiveRecord adapter has to call it too, which it_behaves_like 'a policy repository' enforces.
  • Four contract examples covering the compile, the passthrough, the clearing, and the applies-to-all program. Their fixture scope_rego is corrected to package gitlab.scope, the package the transpiler actually emits.
  • The transpiler is based on the gitlab-policy-to-rego TypeScript PoC, with identifiers spelled out, including the Rego loop variables (framework_id, not fw), because scope_rego is what someone reads out of the row when investigating a gate decision. The four golden fixtures are regenerated from the transpiler, so any future change to the generated text shows up as a fixture diff.

References

How to set up and validate locally

The gem runs in its own isolated bundle:

cd gems/gitlab-policy-store
bundle install
bundle exec rspec    # 42 examples, 0 failures
bundle exec rubocop  # 14 files, no offenses
# compiled from policy_scope
Gitlab::PolicyStore.create(organization_id: 1, name: "Framework 5", trigger_id: "deployment_requested",
  policy_scope: { "compliance_frameworks" => [{ "id" => 5 }] }).scope_rego
# => ... scope_included_0 if { some framework_id in input.compliance_frameworks; framework_id in {5} }

# no scope, so it applies everywhere
Gitlab::PolicyStore.create(organization_id: 1, name: "Unscoped", trigger_id: "deployment_requested").scope_rego
# => ... "applies": true, "reason": "no policy_scope: applies to all projects"

# authored Rego preserved, policy_scope cleared
Gitlab::PolicyStore.create(organization_id: 1, name: "Authored", trigger_id: "deployment_requested",
  policy_scope: { "compliance_frameworks" => [{ "id" => 5 }] },
  scope_rego: "package gitlab.scope\n# mine").policy_scope
# => nil

MR acceptance checklist

No changelog entry: this is a developer-facing, gem-internal change with no callers yet.

Evaluate this MR against the MR acceptance checklist.

Edited by Marcos Rocha

Merge request reports