Create Compatible Model / Platform Catalogue

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

This issue is for work related to creating a catalogue of compatible models and platforms. For additional information on the community contributions framework, see https://gitlab.com/gitlab-org/gitlab/-/issues/548004+s

Key aspects include:

1) Create a centralized place for contributors to share and document model compatibility.

The north star is that the compatible model catalogue should be located at https://gitlab.com/explore/ as a distinct tab from the CI/CD Catalogue. This has been established as the centralized place for all GitLab catalogues, and will include the Agents and Workflow Catalogue, as well as the MCP Catalogue.

The MVP may be located elsewhere. Options include:

  • GitLab Duo Self-Hosted Compatible Models Community Hub at about.gitlab.com/community/
  • a repository with a markdown create merge requests with results, that is reviewed and merged by the CM team. This could be used to create a simple GitLab Page.
  • a page in the Gitlab handbook classified it as community curated feedback
  • an issue capturing compatible models and testing results
  • a tab within GitLab Duo Self-Hosted Documentation under Configuration Types and Authentication; contributors can submit MRs to that page for CM team review

2) Define key fields and requirements for adding a compatible model to the centralized documentation. Any centralized documentation should consider these core fields:

  • Model and Platform Registry: Community-contributed profiles for compatible models and platforms including:
    • Structured:
      • technical specs (model version, any quantization) w/ link to HuggingFace or other
      • hardware used
      • any scripts to adopt to an OpenAI API standard
      • active users
      • AIG deployment method + version
      • GL version
  • Configuration Library: Working templates, testing scripts, and deployment guides maintained by successful implementers
  • Experience Reports: Detailed implementations with trade-off analyses from similar organizational contexts
    • per feature quality (i.e. open benchmark scores, manual testing)
    • speed
    • cost
    • code language support
  • Contributor/Source Tags: Users can easily distinguish between compatible models contributed by:
    • GitLab - GitLab would seed the community with some initial compatible models (approximately 10)
    • GitLab Partner Verified creators
    • non-verified community contributors
  • Knowledge Base
    • Troubleshooting Repository: Searchable solutions database built from community problem-solving
    • Best Practices Guide: Evolving documentation based on collective implementation experience

Definition of Done

  • There is a centralized place for GitLab Self-Hosted users to browse compatible models and platforms.
    • Compatible model catalogue is live at https://gitlab.com/explore/ OR approved MVP location is operational
    • Navigation and discoverability from main GitLab properties and GitLab Duo Self-Hosted documentation is implemented
  • Structured data schema implemented for all required fields
  • Partner verification process and badges implemented with a clear visual distinction between contributor types
  • Users can search and filtering within the catalogue
  • GitLab Self-Hosted users can contribute their own models and platforms validated as compatible for GitLab review and approval.
    • Merge request template for model submissions created
    • Custom Models team approval workflow configured
    • Quality control standards and review checklist established
Edited by Susie Bitters