Loading
Schedule and paginate Claude Code session list ingestion
What does this MR do and why?
Pulls Claude Code sessions from the Claude Compliance API (Anthropic) every 6 hours and records them as AI governance sessions. Behind the claude_code_compliance_api_ingestion feature flag (wip, default off, checked per top-level group).
ScheduleIngestionWorker(cron, every 6 hours) starts one session-list walk per enabled integration. Jobs go out in waves of 10, at most 5 minutes apart, so they don't all hit Sidekiq at once (same fix as !262205 (merged)). Each job keeps its namespace in the logging context.SyncSessionsWorkerreads one page of up to 500 sessions updated in the last 12 hours, then enqueues itself for the next page. For eachclaude_codesession it finds the GitLab user by email (UserResolver) and records it with the existingSessions::CreateService. Sessions it can't attribute are dropped and logged.- When a walk reaches the last page, it sets
last_successful_sync_at. A failed walk is not retried (retry: 0); the next run lists the same window again. - Worker settings:
concurrency_limit10 andurgency :throttled(as in !262205 (merged)). UserResolvernow logs a missing Anthropic email asno_anthropic_email.
Database queries
ai_compliance_anthropic_integrations is empty on GitLab.com, so plans were run after inserting 2,000 integrations on the Database Lab clone. All use an index and return rows.
| Query | Where | Plan |
|---|---|---|
Batch start, boundary, rows (enabled scope) |
schedule_ingestion_worker.rb:33, integration.rb:29 |
start, boundary, rows |
| Namespace and route preload | integration.rb:30 |
namespaces, routes |
Load integration, set last_successful_sync_at |
sync_sessions_worker.rb:32, :46 |
select, update |
SQL
SELECT "ai_compliance_anthropic_integrations"."id" FROM "ai_compliance_anthropic_integrations" WHERE "ai_compliance_anthropic_integrations"."enabled" = TRUE ORDER BY "ai_compliance_anthropic_integrations"."id" ASC LIMIT 1;
SELECT "ai_compliance_anthropic_integrations"."id" FROM "ai_compliance_anthropic_integrations" WHERE "ai_compliance_anthropic_integrations"."enabled" = TRUE AND ("ai_compliance_anthropic_integrations"."id" >= 1) ORDER BY "ai_compliance_anthropic_integrations"."id" ASC LIMIT 1 OFFSET 100;
SELECT "ai_compliance_anthropic_integrations".* FROM "ai_compliance_anthropic_integrations" WHERE "ai_compliance_anthropic_integrations"."enabled" = TRUE AND ("ai_compliance_anthropic_integrations"."id" >= 1) AND ("ai_compliance_anthropic_integrations"."id" < 143);
SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."id" IN (6543, 9970, ...);
SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Namespace' AND "routes"."source_id" IN (6543, 9970, ...);
SELECT "ai_compliance_anthropic_integrations".* FROM "ai_compliance_anthropic_integrations" WHERE "ai_compliance_anthropic_integrations"."id" = 500 LIMIT 1;
UPDATE "ai_compliance_anthropic_integrations" SET "updated_at" = '2026-10-22 12:00:00', "last_successful_sync_at" = '2026-10-22 12:00:00' WHERE "ai_compliance_anthropic_integrations"."id" = 500;Out of scope
- Transcripts: https://gitlab.com/gitlab-org/gitlab/-/work_items/623178
- Internal Events for drop reasons: !263844 (stacked on this MR)
How to set up and validate locally
bundle exec rspec ee/spec/workers/ai/compliance/anthropic ee/spec/lib/ai/compliance/anthropic/user_resolver_spec.rb ee/spec/models/ai/compliance/anthropic/integration_spec.rbOn GDK, with an enabled integration that has a real Compliance Access Key:
Feature.enable(:claude_code_compliance_api_ingestion, group)Ai::Compliance::Anthropic::SyncSessionsWorker.new.perform(integration.id)- Check the new
Ai::Governance::Sessionrows and the integration'slast_successful_sync_at.
References
- Issue: https://gitlab.com/gitlab-org/gitlab/-/work_items/623177
- Rollout: https://gitlab.com/gitlab-org/gitlab/-/work_items/623179
- Design doc: gitlab-com/content-sites/handbook!20750 (merged)
- Epic: https://gitlab.com/groups/gitlab-org/-/work_items/21540
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist.
Edited by Hitesh Raghuvanshi