Publish skills on push to the default branch

What does this MR do and why?

This MR adds the sidekiq job that kicks off the scanners for skills that we introduced in previous MRs. I chose sidekiq so we didn't block any other work that depended on the default branch publishing. This is very much an async task.

It also adds the ai_catalog_skills WIP feature flag which this MR is gated behind. The flag is project scoped because scoping it to a user will cause all sorts of hell. This means the rollout will be a little different to usual and "Can you enable this flag for me?" will have to be answered with my favourite phrase, "No".

It's a fairly straight-forward sidekiq job with the following setup:

1. Low urgency: If this takes ages, it's no big deal. The catalog data is just out of sync for a short while.
2. defer if unhealthy: If the catalog items or catalog item versions tables are unhealthy, pop back in 5 minutes and try again. Similar to the low urgency, we don't need to further gum up an unhealthy table. This can wait.
3. Retry on lease contention: PublishService holds a per-project lease, so only one publish runs per project at a time. If a second push turns up mid-publish it loses the lease, and rather than shrugging and dropping it we raise and let sidekiq retry 30 seconds later, up to 25 times. Without that, the push that lost the race just never gets published, because the publish that won may have read the tree before that commit landed.
4. Licensed only: the hook and the worker both check Project#ai_catalog_available?, the same predicate the policies use, so free projects never get a job or an item.

Known gap

The first publish to a branch is not technically an update, so when a project is created, this job is not ran. This is known and will be fixed in a future update.

References

• Closes #630879
• Epic: gitlab-org#22961
• Proof of concept merge request: !257276
• Depends on the Publish slice: #630878
• Rollout issue: #630885

Screenshots or screen recordings

No UI changes.

How to set up and validate locally

1. Enable the flag for a project in the Rails console: Feature.enable(:ai_catalog_skills, Project.find(<id>))
2. In that project, commit skills/my-skill/SKILL.md on the default branch with YAML front matter containing name: my-skill and a description, and push.
3. Check the Catalog Item was created: Ai::Catalog::Item.where(project_id: <id>, item_type: :skill) in the Rails console. The AI Catalog UI does not list skills until the browse slice merges, so check in the console.
4. Edit the SKILL.md and push again to see a new Item Version created.
5. Repeat the previous step as many times as you like. Try making invalid versions to see that only valid versions get published.
6. Delete the SKILL.md and push to see the Catalog Item soft-deleted.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.