Draft: Repository-backed skills in the AI Catalog (proof of concept, do not merge)

What does this MR do and why?

Proof of concept. Do not merge. This branch exists only to share code. It is not intended to ship in any form.

This adds a repository-backed skill item type to the GitLab AI Catalog. A skills/<name>/SKILL.md file in a project repository is found by a scan and published as a catalog item. The content stays in git; the catalog stores metadata plus a pointer to the file.

It also adds a scaffolder. The create page writes the first SKILL.md to a new branch and opens a merge request, and stops there. It is never an editor: editing a skill is a link to the Web IDE. This is the first time the catalog UI writes to git rather than reading from it, which is where the repo-backed premise gets properly tested.

It does two jobs. It de-risks repository-backed authoring for the wider epic, and it produces something runnable to poke at. The design document "Repository-Backed Authoring for the AI Catalog" never mentions skills and leaves the write path unspecified, so this fills real gaps rather than re-treading the document. It also answers that document's Open Question 1, on whether in-app sync status can replace the CI job.

Walkthrough

📺 Watch on YouTube

Findings

Open to show findings. Warning, needs triaging.

Finding 1: the existing write path cannot express a publish

The publisher writes ActiveRecord directly rather than going through Ai::Catalog::Items::BaseCreateService. This is not a shortcut. There are five structural reasons:

1. There is no create-or-update entry point. BaseCreateService only creates and BaseUpdateService only updates.
2. BaseUpdateService#execute returns an error when nothing changed. That is the common case on every re-publish.
3. BaseCreateService:10 requires Ability.allowed?(current_user, ...). A publish has no user.
4. BaseCreateService:41 hardcodes version: DEFAULT_VERSION and cannot express a bump.
5. BaseCreateService:43 always sets release_date, and ItemVersion#validate_readonly:145 then freezes the row. That forces a new version on every later publish.

Both existing non-form writers, Flows::SeedFoundationalFlowsService and Gitlab::Ai::Catalog::ThirdPartyFlows::Seeder, already bypass the service layer for the same reasons. Repository-backed authoring would be the third. The epic needs an idempotent create-or-update entry point, built before repo-backed authoring can reuse the write path.

Finding 2: version churn

Three typo fixes to one SKILL.md produced v1.0.0, v1.1.0, v1.2.0 and v1.3.0, all immediately released.

Every publish that changes content mints a new released version and rewrites latest_released_version_id. A consumer pinned to 1.x silently follows every edit, so pinned_version_prefix gives no stability. The ai_catalog_item_versions table was designed for versions someone chose to publish. Driving it from incidental merges is a mismatch the epic has to resolve.

Finding 3: commit SHA vs blob SHA

The design document's data mapping puts commit_sha on the version for provenance. In this PoC the definition jsonb is the change-detection surface, so storing a commit SHA would mint a new version on every publish where the default branch had moved for any unrelated reason.

The definition stores the git blob object ID instead. It is content-addressed, so it serves as both provenance and a precise change key, and it is what makes repeated publishing a no-op.

Finding 4: renaming a skill destroys it

Identity is the directory name, because the Agent Skills specification has no id field and this PoC adds no internal_id column. The design document deliberately uses a frontmatter id so that "File renames and moves are safe as an item's id survives file reorganization".

Running git mv skills/cosign-blob skills/cosign with byte-identical content gave:

• old item id=33, deleted_at set, 4 versions
• new item id=35, 1 version, v1.0.0

Version history and any enablement are lost. This is accepted for the PoC and not fixed. It needs a decision before anything like this ships.

Finding 5: the create mutation cannot return the thing it created

Every other create mutation in ee/app/graphql/mutations/ai/catalog/ returns field :item, <SomeType>. AiCatalogFlowCreate and AiCatalogAgentCreate both do. A repo-backed skill has no catalog item at create time. The item appears later, when the merge request is merged and the publisher runs.

So the new mutation is named AiCatalogSkillScaffold, not AiCatalogSkillCreate, and returns field :merge_request, Types::MergeRequestType. There is precedent for that payload. Mutations::Security::Finding::CreateMergeRequest (ee/app/graphql/mutations/security/finding/create_merge_request.rb:11-14) is the only other mutation that does branch, commit and merge request and returns a MergeRequestType.

The knock-on effect reaches the frontend. The shared create page ee/app/assets/javascripts/ai/catalog/pages/ai_catalog_item_new.vue reads item.id and redirects to that item's show page (lines 81-83). With no item, it cannot be reused. The scaffolder has its own page and navigates the user out of the catalog entirely, to the merge request. This is the clearest signal so far that repo-backed authoring does not fit the catalog's existing CRUD model, and it is worth capturing regardless of what ships.

Finding 6: creation is gated by repository permissions, not catalog permissions

Agents and flows gate creation on the create_ai_catalog_agent and create_ai_catalog_flow abilities. Scaffolding a skill is a push, so the mutation authorizes push_code and declares granular token permissions [:push_code, :create_branch, :create_merge_request]. Two permission models that do not line up.

There is a concrete wrinkle. ai_catalog_nav_actions.vue (lines 45-51) gates every "New" button on canAdmin, which is glAbilities.adminAiCatalogItem, before it consults the per-type canCreateFn. A user with push access but without catalog admin cannot reach the form from a project. Not fixed here, because changing that shared component affects agents and flows too.

Finding 7: the specification has no display name

The scaffolder now conforms to the Agent Skills specification. That settles the slug question from an earlier revision of this description, and replaces it with a sharper one.

The specification requires name to match the parent directory name, and restricts it to lowercase alphanumerics separated by single hyphens, at most 64 characters. So there is no separate display name to derive a slug from. The two form fields collapsed into one, and the mutation takes a single name.

The casualty is the catalog list. Items are now named cosign-blob rather than Cosign Blob, because the specification defines no field for a human readable name. Agents and flows have proper names; repo-backed skills cannot, without either departing from the specification or using its metadata map:

metadata:
  display-name: Cosign Blob

metadata is explicitly provided by the specification for "additional properties not defined by the Agent Skills spec", so using it would not be inventing a key. It would mean the publisher preferring metadata['display-name'] over name when setting the item name. Not done here, but Finding 11 takes exactly that route for visibility, so the mechanism is proven and this is a small follow-up if the catalog listing matters.

Worth noting this also removes the rename hazard an earlier revision flagged. Name and directory are now the same thing by definition, so they cannot drift apart. Finding 4 still stands: renaming that one value still destroys and recreates the item.

Finding 8: the scaffolder can write a file the publisher will ignore

Gitlab::Ai::Catalog::Skills::RepositoryScanner::MAX_SKILLS is 100. The scanner sorts paths and takes the first 100.

Observed on the GDK fixture project, which already holds 100 skills. Scaffolding skills/manual-smoke-test/SKILL.md succeeded, committed correctly and opened a merge request. The file sorts at index 100, exactly one past the cap, so the publisher would silently skip it.

The scaffolder cannot reasonably predict this. It does not know what the repository will look like when the merge request merges. Not fixed. It is an argument for the sync-status surface this PoC has been probing.

Finding 9: duplicate scaffolds are not detected

If skills/<slug>/SKILL.md already exists on the default branch, the service rejects it before creating a branch, with the message skills/<slug>/SKILL.md already exists on master.

Open merge request branches are not scanned. Two people scaffolding the same slug get two merge requests, and the second one hits an ordinary merge conflict. Scanning open merge request source branches was judged not worth the cost for a PoC, and it would still be racy.

Finding 10: there is no event to subscribe to for a push

The trigger had to react to "a SKILL.md changed on the default branch". There is no Gitlab::EventStore event for a git push. Repositories::DefaultBranchChangedEvent fires when the default branch setting changes, not when someone pushes to it, and no Git::BranchPushedEvent exists.

So the only route is an EE prepend on Git::BranchPushService plus a worker that recomputes the changed paths itself with Gitlab::Git::Push#modified_paths. Six features in ee/app/services/ee/git/branch_push_service.rb already do this, including Elasticsearch, Zoekt and repository X-ray. The established alternative, Git::BranchHooksService#commit_paths, is not usable: it caps at 100 commits and makes one Gitaly call per commit.

Publishing on push rather than on merge is deliberate, and turns out to be the better fit. A merge into the default branch is a push, so merged scaffolder merge requests are covered, and so are direct commits and deletions that a merge hook would miss. Deleting a SKILL.md with a direct commit correctly soft-deletes the item.

The cost is that every default-branch push on every project enqueues a job, because no cheap check can tell whether a project cares about skills. The first SKILL.md a project ever commits is exactly the case that has to work, so there is nothing to gate on. The job does one diff_stats call and usually returns immediately, which is fine for a proof of concept and unacceptable at GitLab.com scale. This is the concrete argument for the design document's project_settings.ai_catalog_publishing_enabled opt-in, which this branch does not have.

One known gap: paths cannot be computed when a branch is created, so a project whose very first push lands SKILL.md files does not publish until the next push.

Finding 11: the metadata map is where catalog concerns can live

Visibility was the last thing about a repository-backed skill that lived only in the database. The publisher hardcoded private, nothing could change it, and it sat as a mutable catalog-side field on an otherwise immutable item. An earlier revision of this description listed that as unresolved.

The specification has no visibility field, but it defines metadata as a free-form string map for "additional properties not defined by the Agent Skills spec". That is the sanctioned extension point, so visibility now lives there:

metadata:
  visibility: public

An absent metadata block means private, so hand-written skills keep working and the key stays optional. An unrecognised value also means private and is logged: a typo in an optional field should not drop the whole skill, and private is the only default that cannot leak anything by accident. The create form gets the same visibility control as agents and flows, confirmation modal included.

This closes the loop. Every field the catalog holds for a skill now comes from the repository, and editing the metadata block and merging changes it. Verified by flipping a published skill from public to restricted with a direct commit.

One asymmetry remains, and it belongs to the database rather than to this design. Item#validate_visibility_cannot_be_reduced refuses to pull an item back from public while it is enabled elsewhere, so a reduction in YAML can be rejected. The publish reports it as a failure rather than leaving the repository and the catalog quietly disagreeing, but it is a case where git is no longer the source of truth. Anything built on this pattern needs an answer for a repository that asks for something the catalog cannot grant.

Finding 12: attribution assumes a user session performed the write

Every published skill rendered as "Created by Unknown". PublishService has no current_user, so it never set created_by, and ai_catalog_item_versions.created_by_id is nullable with no model validation to catch the omission.

The two other non-form writers, SeedFoundationalFlowsService and Gitlab::Ai::Catalog::ThirdPartyFlows::Seeder, omit created_by as well. Nobody noticed, because both produce gitlab_maintained items and ai_catalog_item_user_attribution.vue:45 short-circuits those to render "GitLab" before reaching the fallback. Skills are the first non-form-written items that are not GitLab-maintained, so a long-standing omission became visible rather than being introduced. That is the same shape as Finding 1: the write path assumes a form submission, and everything that is not one has to work around it.

The fix takes the provenance git already has. A repository-backed item has no session to attribute a write to, but it does have a commit, so the version author is whoever authored the commit that produced the blob. That survives a merge, because a merge commit does not touch the path and a squash carries the author over, which makes the attributed user the person who filled in the form. Only resolved for skills that actually changed, so an unchanged publish costs nothing and a cold publish costs one Gitaly call per skill.

That is only half the problem, and the other half is the more interesting one. ai_catalog_item_versions can only store a User foreign key, but git attribution is a name and an email, and the email need not belong to a GitLab account. Every external contributor, every bot commit, and anyone whose git config email is not on their profile resolved to nothing and rendered as "Created by Unknown", while the repository browser, reading the same commit, displayed the author's name perfectly well.

So the version definition now records commit_sha and author_name alongside blob_sha, and the attribution component falls back to the name when there is no user to link to. A matched user still wins, because it can be linked. The detail page also links the commit, which is the write that actually produced the version.

Recording commit_sha is safe because change detection compares blob_sha explicitly, not the whole definition. Finding 3 is about not using a commit SHA as the change key, which would mint a version on every unrelated push. Storing it purely as provenance does not.

Two limits remain. Versions published before this change keep a null author for good; they pick up one only when their content next changes and mints a new version. And the catalog still models attribution as one identity per version, because a database-authored item has exactly one. Git has author and committer, co-authors, and a full history per line. A repository-backed item can support richer provenance than the catalog can represent.

Deviation from the design document

The design document's "Item duplication" work stream describes almost the same flow: create a branch, write the file, open a merge request. It specifies doing the work asynchronously "to take load away from the web nodes", with a toast and a todo to notify the user.

This is synchronous instead. It is a PoC, and returning the merge request URL immediately is worth more than the load saving. Synchronous commit-from-a-request is also well precedented, by ComplianceManagement::Projects::CreateCiConfigService and by the ConfigureSast and ConfigureSecretDetection mutation family. At real catalog scale the design document is probably right.

Experiment results

Run against a GDK fixture project holding 100 skills.

Deletion reconciliation works. Removing a SKILL.md soft-deletes the item. A simulated Gitaly outage makes the scan return nil, the publish returns reason: :scan_failed, and the live item count is unchanged. This matters because reconciling deletions against an unknown state would delete a project's whole catalog on a transient error.

Idempotency works. Three consecutive publishes with no repository change report created=0 updated=0 unchanged=3 and create no versions.

Timings for 100 skills in one project:

Operation	Time
Scan (Gitaly + parse)	192 ms
Full publish, cold	1544 ms
Full publish, no-op	124 ms

Known gaps

• Nothing in GitLab reads AI Catalog skill enablement. The Enable button renders and writes an ai_catalog_item_consumers row, but no consumer exists. Cross-project consumption is follow-up work owned by other groups.
• Skills are not governed by any Duo setting, so unlike agents and flows they cannot be turned off centrally. isItemTypeEnabledInDuoSettings hardcodes true. That is the governance work, out of scope here. Note this is separate from per-item visibility, which Finding 11 resolves.
• ItemConsumersFinder and ProjectItemsFinder derive their type lists from the model enum, so callers that do not pass explicit item_types now see skills alongside agents and flows.
• The scaffolder writes new skills only. Duplicating an existing skill into another project, which is what the design document's duplication work stream actually describes, is not built.

What was built

• A skill item type, enum value 5, with no migrations. It uses the existing definition jsonb to carry {path, git_ref, blob_sha}.
• Gitlab::Ai::Catalog::Skills::RepositoryScanner, a single Gitaly regexp scan for skills/<name>/SKILL.md on the default branch, with YAML front matter parsing. Modelled on the existing Gitlab::DuoWorkflow::WorkspaceAgentsReader, which already does this for .agents/agents/*.md.
• Ai::Catalog::Skills::PublishService, a declarative desired-state reconcile modelled on Ai::Catalog::FoundationalFlows::SyncService, with an exclusive lease per project.
• AiCatalogSkill and AiCatalogSkillVersion GraphQL types.
• List and detail pages, in both the Explore catalog and the project and group Automate area. The detail page links out to the Web IDE for edits.
• Ai::Catalog::Skills::ScaffoldService, which commits skills/<directory>/SKILL.md to a new branch and opens a merge request. Branch creation is folded into the commit by giving Files::CreateService a branch_name that differs from start_branch, so there is one Gitaly call and no orphan branch if the commit is rejected. Branch names come from repository.next_branch.
• The AiCatalogSkillScaffold mutation, which returns the merge request rather than a catalog item.
• A create page and form. It captures a name, a description, a visibility and the markdown body, and lists every file it will create before submit. Visibility uses the same control as agents and flows.
• Scaffolded directories conform to the Agent Skills specification: SKILL.md with name and description front matter plus a metadata.visibility key, and empty scripts/, references/ and assets/ directories, each holding a .gitkeep so git tracks them.
• An "Edit in Web IDE" link on the detail page, using the existing ideEditPath field on Repository::BlobType. Editing a skill is a link, never a form.
• Ai::Catalog::Skills::PublishWorker, which republishes a project after a push to its default branch touched a SKILL.md. Wired up by an EE prepend on Git::BranchPushService, following Security::SyncLinkedPipelineExecutionPolicyConfigsWorker.
• A rake task, kept so the scan can still be run on demand and observed.

Tests

All pass locally: publish and scaffold service specs, scanner specs against a real repository, GraphQL request specs including granular token authorization, and sidebar menu specs. On the frontend, 3151 Jest tests pass.

Two Capybara specs drive the skill pages in a browser, against a real repository and a real Apollo client. ai_catalog_skill_details_spec.rb covers the detail panel and the Web IDE link. ai_catalog_skill_create_spec.rb fills in the create form, submits it, and asserts on the committed file and on the merge request that opens.

These exist because the generic ai_catalog_item_details_spec.rb only asserts that an item name renders. It never exercises the skill-specific components, which is how two real bugs shipped behind a green suite earlier in this branch. No catalog item exists at all in the create flow, so nothing else in the suite would notice if the form stopped writing to git.

References

• Epic: gitlab-org#22961
• Design document MR: gitlab-com/content-sites/handbook!18859 (merged)
• Governance settings issue: #607798
• Competing proposal that needs a PM decision: #600470
• Agent Skills specification: https://agentskills.io/specification

Screenshots or screen recordings

Screenshots still need to be added manually, as this MR changes the UI.

Before	After

How to set up and validate locally

1. Create a project and commit some skills, for example skills/cosign-blob/SKILL.md with YAML front matter containing name and description.
2. Run bundle exec rake "gitlab:ai_catalog:publish_skills[group/project]". Pushing those commits to the default branch also publishes them automatically; the rake task is kept so the scan can be run on demand.
3. Browse /explore/ai-catalog/skills and <project>/-/automate/skills.
4. To see the version churn finding, edit a SKILL.md, re-run the task, repeat, then inspect ai_catalog_item_versions.
5. To see the rename finding, git mv a skill directory and re-run the task.
6. To use the scaffolder, go to /explore/ai-catalog/skills or <project>/-/automate/skills and select New skill. Fill in the form and submit. You land on the merge request it opened. Merge it, and the skill appears in the catalog on its own.
7. To check specification conformance, scaffold a skill and inspect the branch. You should get SKILL.md plus three .gitkeep files, and the name key should equal the directory name.
8. To see visibility round-trip, scaffold a public skill, merge it, then edit metadata.visibility to restricted on the default branch. The catalog item follows.
9. To see the collision handling, scaffold the same name twice. While the first merge request is open you get a second one. Once it is merged, the next attempt is rejected.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability. Please evaluate this MR against the MR acceptance checklist.

The checklist is only partly applicable here, as this MR is not intended to merge.