Sync dedicated SAs roles to be in line with the flow's role when it changes

What does this MR do and why?

In this preceding MR: !253659 (merged), provisioning matches an autonomous trigger's dedicated account to the role the flow's own account holds, but nothing kept the two matched afterwards if the flow's SA role changes.

Current problem

  • Demotion: the flow account loses access, the dedicated account keeps it, and an unattended run executes with more project access than the flow it is running. Nothing surfaces it.
  • Promotion: the dedicated account is left short, runs fail, and after three consecutive failures record_failure! sets active: false and emails the schedule owner. An unrelated membership edit quietly switches someone's schedule off.

Solution

This reconciles the dedicated SA's role against the flow's, immediately before a scheduled run, ensuring both SAs continue to be in sync when the flow SA's role changes.

Why at the run, and not on a timer or an event

Nothing in the codebase observes a change to the flow account's role. The value being mirrored is effective access, i.e. max_member_access, which reads project_authorizations and is the maximum across project membership, group membership and group shares. No single row owns it, so no single event means "the role changed".

Re-reading the value avoids having to enumerate its causes. Doing it at the run costs two indexed reads and closes the window to zero at the only moment the account can act, i.e. Ai::RunFlowScheduleWorker is the sole producer of trigger_source: :scheduled, and the account's only token path runs through it.

Behavioural Decisions / Approaches

1. A refused reconcile is handled by direction

Direction Behaviour Rationale
Promotion fails Run proceeds Under-privilege is not a safety problem but the run fails on its own merits
Demotion fails Run refused and failure recorded against the schedule Otherwise the account acts with more access than the flow

2. Editing a trigger reconciles too, opportunistically

When an edit repoints the trigger at a different flow, and the reconciliation fails, it is deliberately not fatal. Instead we simply log the error and let the worker run reconciliation enforce it. This prevents a membership problem the editor did not cause and cannot fix (e.g. a seat cap, a role inherited from an ancestor group, etc.) causing a roll back to an unrelated change like a trigger rename.

3. Membership writes are authorized

Both paths go through the authorized member path, matching the no skip_authorization approach during provisioning. An edit is capped by what its editor may assign (including the role the account currently holds, since prevent_role_assignement? refuses to change a member whose existing role the actor could not assign). A run carries no actor, which Members::CreatorService already treats as a system write.

Alternative approaches considered

  • Nightly sweep over all triggers.
    • It does O(N) work for what might not actually need change (i.e. assumption: editing a flow's SA role is rare or will be low in terms of numbers).
    • Cadence dependent: at nightly cadence it loses the race it was meant to win, e.g. an hourly schedule burns three failures in three hours and deactivates long before 03:00. Run-time reconcile simplifies it.
  • Members::UpdatedEvent subscriber. Would need four or more subscriptions (project and group sources, destroy, shares, transfer) and would still miss every CreatorService write, which is an incomplete approach.

How to set up and validate locally

Prerequisites

  • Duo Enterprise enabled
  • GDK running with all services up
# Setup

project = Project.find(1000000) # replace with project_ID
user = User.find(1) # replace with user_id

# FFs
Feature.enable(:autonomous_service_account_execution, project)
Feature.enable(:ai_flow_schedules)

# Create scheduled event type
scheduled = Ai::FlowTrigger::EVENT_TYPES[:scheduled]

# Get flow's SA
consumer = Ai::Catalog::ItemConsumer.where(project: project).find { |c| c.active_service_account.present? }
flow_sa  = consumer.active_service_account

# project_authorizations is refreshed asynchronously, so every membership change
# needs a recalculation or reads lag behind causing unrelated failures.
refresh = ->(u) do
  AuthorizedProjectUpdate::ProjectRecalculatePerUserService.new(project, u).execute
  project.team.purge_member_access_cache_for_user_id(u.id)
  project.project_members.reset
end

set_flow_role = ->(level) do
  project.member(flow_sa).update!(access_level: level)
  refresh.call(flow_sa)
  project.team.max_member_access(flow_sa.id)
end

# Compares what the service compares: the flow's EFFECTIVE access against the
# dedicated account's member row, which is the only thing we write.
levels = ->(t) do
  dedicated = User.find(t.reload.autonomous_service_account_id)
  [flow_sa, dedicated].each { |u| refresh.call(u) }

  { flow_effective: project.team.max_member_access(flow_sa.id),
    dedicated_row:  project.member(dedicated.id)&.access_level }
end

# Edit the flow trigger with scheduled event
relevel = ->(t, desc) do
  Ai::FlowTriggers::UpdateService
    .new(project: project, current_user: user, trigger: t.reload)
    .execute(ai_catalog_item_consumer_id: consumer.id, event_types: [scheduled], description: desc)
end

worker    = Ai::RunFlowScheduleWorker.new
reconcile = ->(t) { worker.send(:reconcile_service_account_role, t.reload, project) }

set_flow_role.call(Gitlab::Access::DEVELOPER)   # => 30, known starting point

# Step 1: create the trigger

result = Ai::FlowTriggers::CreateService
  .new(project: project, current_user: user)
  .execute(
    ai_catalog_item_consumer_id: consumer.id,
    event_types: [scheduled],
    description: 'reconcile check',
    filter: { 'scheduled' => { 'frequency' => 'DAILY', 'hour' => 3, 'minute' => 0 } }
  )

trigger = result.payload
schedule = trigger.flow_schedules.first
dedicated_id = trigger.autonomous_service_account_id

{ ok: result.success?, error: result.message, levels: levels.call(trigger), schedule: schedule&.id } => # Provisioning matched the flow's role. Expect => ok: true, flow_effective: 30, dedicated_row: 30.  

# Step 2: trigger edit path follows the flow SA's role in both directions

# Promote the role
set_flow_role.call(Gitlab::Access::MAINTAINER)
relevel.call(trigger, 'after promotion').success?
levels.call(trigger) # => 40 / 40

# Demote the role
set_flow_role.call(Gitlab::Access::REPORTER)
relevel.call(trigger, 'after demotion').success?
levels.call(trigger) # => 20 / 20

# Step 3: Owner is never mirrored

# Promote the role
set_flow_role.call(Gitlab::Access::OWNER)
relevel.call(trigger, 'owner check').success? # => true, the edit still succeeds
levels.call(trigger) # => 50 / 20, unchanged

# Reset role back to DEVELOPER
set_flow_role.call(Gitlab::Access::DEVELOPER)
relevel.call(trigger, 'back to developer')
levels.call(trigger) # => 30 / 30

# Step 4: the run reconciles before executing - The main focus of the MR change. 

# Drift created with no edit at all
set_flow_role.call(Gitlab::Access::REPORTER)
levels.call(trigger) # => 20 / 30   <- real drift, nothing has noticed

reconcile.call(trigger) # => false, the run may proceed
levels.call(trigger) # => 20 / 20, dedicated SA catches up

# Then the full path, where only the worker touches the account:
set_flow_role.call(Gitlab::Access::DEVELOPER)
levels.call(trigger) # => 30 / 20 <- drifted again, dedicated SA left behind

Ai::RunFlowScheduleWorker.new.perform(schedule.id)
levels.call(trigger) # => 30 / 30   <- the run reconciled it and the dedicated SA catches up

schedule.update!(consecutive_failure_count: 0, active: true)

# Step 5: a blocked demotion stops the run

dedicated = User.find(dedicated_id)

# 1. Precondition: dedicated SA at 30, no floor in the way
project.group.members.find_by(user_id: dedicated_id)&.destroy!
set_flow_role.call(Gitlab::Access::DEVELOPER)
reconcile.call(trigger)
levels.call(trigger) # => 30 / 30

# 2. Floor above the level we are about to demote to
project.group.add_member(dedicated, Gitlab::Access::MAINTAINER)
refresh.call(dedicated)
project.group.members.find_by(user_id: dedicated_id).access_level # => 40, must not be nil

# 3. Drop the flow below the floor
set_flow_role.call(Gitlab::Access::REPORTER)
levels.call(trigger) # => 20 / 30

# 4. The demotion is now refused by higher_access_level_than_group
reconcile.call(trigger) # => true: the run must not proceed

# Through the full worker:
Ai::RunFlowScheduleWorker.new.perform(schedule.id)
schedule.reload.last_run_error
# => "The service account holds more access than the flow, and could not be adjusted"

# Step 6: a reconcile failure does not block an edit

# Floor still in place from step 5:
r = relevel.call(trigger, 'non-fatal check')
{ ok: r.success?, description: r.payload&.description } # => { ok: true, description: "non-fatal check" }

# Step 8: Cleanup

project.group.members.find_by(user_id: dedicated_id)&.destroy!
refresh.call(User.find(dedicated_id))
Ai::FlowTriggers::DestroyService.new(trigger: trigger.reload, current_user: user).execute
set_flow_role.call(Gitlab::Access::DEVELOPER)

References

Screenshots or screen recordings

Before After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Shola Quadri

Merge request reports

Loading
Loading