Skip to content

Extend the /projects API to support filtering by language

The Rust infrastructure team maintains a project called Crater, which is used by the Rust compiler developers to ensure no regression is present in our releases. Every time an experiment is created, Crater fetches all the public Rust code available on crates.io (Rust's package repository) and GitHub and builds it on our infrastructure with the new compiler. Crater is also used to check if breaking changes are feasible.

We would love to add GitLab.com support in Crater, to give public Rust projects hosted on your platform the same benefits as the ones hosted on GitHub.

Unfortunately, I looked at the GitLab API and it's not feasible for us to scrape the list of projects without hammering your server with requests: /api/v4/projects doesn't allow to filter by language, and we would have to call /api/v4/projects/:id/languages for each of them, and that's n+1 queries. There are two ways this could be solved:

  • Include language statistics in the output of /api/v4/projects, so we can filter this on the scraper side (preferred)
  • Add another argument to /api/v4/projects to filter by language on the server side

Could one of those solutions implemented? It's the only thing blocking us to implementing GitLab support in Crater.
Tracking issue on our scraper's repo