<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=419738) </details> <!--IssueSummary end--> ## About When a new group is connected to Jira, we [sync some historical data](https://gitlab.com/gitlab-org/gitlab/-/blob/5747fb0e5b12cc1d3df367834206973ddb2894a1/app/workers/jira_connect/sync_project_worker.rb#L24-25) per project in the group, but we do not sync historic commit data, [documented here](https://docs.gitlab.com/ee/integration/jira/connect-app.html#gitlab-data-synced-to-jira). ## Technical problem There are some known N+1 performance problems in the `Atlassian::JiraConnect::Serializers::CommitEntity`, which is that for every commit we serialize to sync with Jira, we perform [multiple additional Gitaly calls](https://gitlab.com/gitlab-org/gitlab/-/blob/716ac5abc038274b08410f7d43dd2181e2efda80/lib/atlassian/jira_connect/serializers/commit_entity.rb#L24-36). There are some outstanding issues to introduce new Gitaly RPC methods that we could use to get certain data from Gitaly in a single call, rather than a call per commit: - https://gitlab.com/gitlab-org/gitaly/-/issues/3375+ - https://gitlab.com/gitlab-org/gitaly/-/issues/3374+ Those N+1 performance problems are currently marked as blockers for this issue. ## Proposal Although the Gitaly RPC issues are marked as blockers, we could possibly make unperformant N+1 calls to Gitaly if we limit the number of commits we sync, for example only the latest `400` commits made to the repository. This would work if: - Limiting the initial sync of data to the latest `400` commits would still be useful for our customers. - Making up to ~1,200 Gitaly calls per project that we sync in a group was okay for ~"Category:Gitaly". We sync all projects in a group, but we [process 1 project per minute](https://gitlab.com/gitlab-org/gitlab/-/blob/2ebad5adb2dd040ce9f4dabb5f6fd8d4d5586e03/app/services/jira_connect_subscriptions/create_service.rb#L50-57). This would mean we would make up to an additional ~1,200 Gitaly calls per minute.