A custom flow definition is not treated as code

Problem to solve

Custom flows have a file-based representation, but that representation is not currently the authoritative lifecycle object. The operational object is an AI Catalog item: GitLab stores its catalog identity and versions, controls visibility and enablement, creates the associated service account and triggers, pins versions for consuming projects, and executes the catalog-published configuration.

A repository file can serve as an input to a synchronization job, but neither .gitlab/duo/flows/<name>.yaml nor ai-catalog-sync makes the repository authoritative for the complete flow lifecycle. The sync job publishes repository content into the AI Catalog; it does not replace the catalog’s records for version adoption, project enablement, triggers, service-account associations, hiding, or deletion.

The problem is the resulting split between authoring and operation. Teams building serious agentic software want repository-native review, Code Owners, CI validation, reproducible publication, and an auditable change history like any other "code". The current product model can execute a catalog item and can be made to consume repository content, but it does not provide a generally supported repository-backed lifecycle in which the repository is the single source of truth for both the definition and its catalog state.

Why it matters

This split creates practical problems:

  • Review and governance are discontinuous. A YAML change can receive merge-request review, but the live catalog item can still have a different version or lifecycle state. A repository sync path is not, by itself, a proof that the catalog copy matches the reviewed commit. The AI Catalog team is tracking approval-rule concerns even for flows managed through the AI Catalog Sync component in https://gitlab.com/gitlab-org/gitlab/-/work_items/577946+.

  • Reproducibility depends on catalog state. AI Catalog versions are immutable, and consuming groups and projects use pinned versions unless they explicitly adopt an update. This is useful for runtime stability, but it means that publishing a new repository revision does not imply that every consumer runs it. As stated in AI Catalog versioning documentation.

  • Lifecycle automation remains separate from content publication. A CI job can synchronize definition content, but creating, updating, enabling, disabling, hiding, deleting, and adopting versions still belong to the AI Catalog lifecycle. Also, GraphQL flow-management operations required a PAT which do come with its own operational difficulties.

  • Definition size constrains content-heavy flows. The AI Catalog schema has a 64KB per-item validation limit. A large-scale custom flow can easily reach to the limit, esp. considering other limitations on flow definition model and constructs.

Background and evidence

  • PREP Agentic Workflow documented the 64KB item limit, the lack of a developer-grade YAML authoring loop, and the need for validation, testing, and CI-oriented tooling in its Phase 2 discussion and action items.
  • Infrasec Review Agent keeps only a thin bootstrap prompt in the flow YAML and has the agent fetch its real instructions from markdown files in the project repo at run time. The stated reasons are that prompts become version-controlled, every review ties to a commit SHA, prompt diffs are ordinary git diffs, and prompt changes do not require a flow redeploy. They accept added latency on every run and a hard dependency on the fetch succeeding first.
  • CX Platform Engineering documented a repository YAML source of intent, a live AI Catalog copy, manual synchronization, and accepted drift risk in its spec-review flow specification.

References and links

  • The AI Catalog backend architecture describes the current database-backed model and how item definitions are authored through the UI and GraphQL API and stored.
  • The proposal in Repository-Backed Authoring for the AI Catalog would make YAML in a repository the source of truth for custom definitions and use CI to publish them. However, it still deliberately keeps PostgreSQL as the store for catalog metadata, version records, enablement, and search, and it treats runtime definition content as a separately published artifact.
  • The design notes that definition storage is changing in Support large YAML definitions in AI Catalog wi... (#591638), while can solve size limitation problem does not by itself change the catalog’s lifecycle authority.