Skip to content

Backend: Batch request calls to Gitaly when fetching `include`

Summary

Currently, we allow 100 includes and each include triggers a Gitaly call. Our docs recommend not going over 30 Gitaly calls per request: https://docs.gitlab.com/ee/development/gitaly.html#toomanyinvocationserror-errors

Proposal

These requests to Gitaly should be batched

Useful data here: &7290 (comment 1168994718) <- Internal only

Implementation

Work Type Description Issue link
NOTE: 🚨 All below issues can be done in parallel
backend Backend: The gitlab-ci.yml is limited to 100 includes #207270 (closed)
backend Backend: Remove N+1 for Gitaly requests when fetching includes #344829 (closed)
backend frontend Improve the error messaging when fetching remote includes are timing out #351168 (closed)
backend Backend: Improve CI Linter performance through parallelizing HTTP calls #351250 (closed)
backend Backend: Caching includes to improve performance when using remote includes #351252
backend Backend: Batch request calls to Gitaly when fetching include 👈 You are here
backend Backend: Group files by projects in config_file_project_validate_access #382751 (closed)
Edited by Mark Nuzzo