Publish repository skills into the AI Catalog

Why are we doing this work

This is the slice where a SKILL.md in a repository becomes a Catalog Item. Slices A, B and C build the parts; this one puts them together and writes to the database.

It is also the first slice that can be demonstrated on its own, because it ships with a rake task. Point it at a project and watch Catalog Items appear. No push hook and no UI is needed yet.

The reconcile behaviour is the part worth reading carefully, and ADR 0001 is the reason for it. A path present and valid is created or updated. A path genuinely absent from the tree is deleted. A path whose scan was incomplete or failed deletes nothing at all. A file that fails validation must never be mistaken for a deleted one — otherwise a typo in YAML front matter would delete a Catalog Item along with every Item Consumer that enabled it, silently revoking a Skill across every project using it.

To state plainly what that means today: this slice lands before the sync state columns, which arrive in slice G. Until then, a Skill that fails validation simply leaves its Catalog Item untouched. It keeps its last good Item Version and its pointer. That is behaviourally the same as Stale — it is just unlabelled and invisible. Slice G is what makes it legible. Nothing unsafe happens in the meantime.

  • This slice's merge request must call out that a Skill's Internal ID is its directory name, so renaming a Skill directory soft-deletes its Catalog Item and loses its Item Versions and Item Consumers. This diverges from the design document, which specifies a front matter id so that renames survive. It is accepted for now and tracked separately, but it should not land unremarked.

No feature flag check here: nothing reaches this code until slice E or F.

Relevant links

  • Epic: gitlab-org#22961
  • Proof of concept merge request: !257276 — read this to see the whole thing end to end. The domain glossary (CONTEXT.md) and both architecture decision records (docs/adr/) live on that branch, not on master.
  • Depends on slice A, the Validator: #630875 (closed)
  • Depends on slice B, the skill item type: #630876 (closed)
  • Depends on slice C, the repository scanner: #630877

Non-functional requirements

  • Documentation: Not required — no user-visible surface yet. The rake task is for developers.
  • Feature flag: Not required — unreachable until slice E or F lands.
  • Performance: An unchanged republish must cost no writes and no commit lookups. create_item and update_item must assign organization: and project: records rather than bare ids, or Item validation loads a record once per Skill.
  • Testing: publish_service_spec.rb, covering all six buckets and each of the three reconcile outcomes, including a truncated and a nil scan.

Implementation plan

  • backend Note when extracting: publish_service.rb on the proof of concept branch is ahead of this slice. It already carries mark_stale, sync_status, sync_metadata, sync_failed_commit_sha and MAX_FAILURE_REASON, all of which belong to slice G, and item.rb there already has the sync_status enum. This slice strips that back rather than copying the file across.
  • backend Add ee/app/services/ai/catalog/skills/publish_service.rb. Take an exclusive lease keyed ai:catalog:skills:publish:<project_id> with a 10 minute TTL and no retries, run the scanner, and reconcile.
  • backend Return ServiceResponse.success with a payload of created, updated, deleted, unchanged, failed and unpublishable buckets. Use error reasons :scan_failed and :lease_taken.
  • backend Abort before reconciling when the scan returns nil, and skip deletions entirely when the scan is truncated. Per ADR 0001, a Publish that did not read the whole tree deletes nothing.
  • backend Mint a new Item Version with next_version_number only when blob_sha or git_ref changed, so an unchanged republish costs nothing.
  • backend Read version attribution from last_commit_for_path: commit_sha, author_name, and created_by (the resolved GitLab user, which may be nil when no user matches the git author). Rescue a Gitaly failure and log it rather than failing the Publish — attribution is cosmetic.
  • backend Build existing_by_path with group_by on latest_version.def_path, and log when one path maps to more than one Catalog Item. That is an internal integrity fault; the code calls it drift, and that word is reserved for this meaning only.
  • backend Assign organization: and project: records rather than organization_id and project_id in create_item and update_item. Item validates organization for presence and the organization match validation reads project.organization_id, so passing ids makes both validations load a record once per Skill. In update_item, assign them only after the changed? check so an unchanged Skill still costs nothing.
  • backend Add a class comment listing why Items::BaseCreateService and BaseUpdateService cannot express a Publish, so the direct ActiveRecord writes read as a decision rather than a shortcut.
  • backend Add a rake task gitlab:ai_catalog:publish_skills[group/project] in ee/lib/tasks/gitlab/ai_catalog.rake, printing each bucket with counts, paths and per-path messages, and exiting 1 on error.
  • backend Add ee/spec/services/ai/catalog/skills/publish_service_spec.rb.
Edited by 🤖 GitLab Bot 🤖