Explore implementation of merge/rebase/squash via Git2Go
The current way we do merges/rebases/squashes is via a temporary worktree, which is quite inefficient as it requires us to create the worktree, checkout files and then finally remove it, causing a lot of I/O. An alternative way to approach this would be libgit2, which is able to do these operations in-memory without the need for any worktree, at least in the conflict-free case.
As both Git2Go as well as Rugged have the problem that they're blocking a complete thread during the lifetime of C calls and as a merge/rebase/squash may take an extended time, calling these functions directly from Gitaly wouldn't work. An alternative that was discussed several times already (see e.g. gitlab#220644 (comment 365078293)), is to implement a separate binary that encapsulates all calls to libgit2 and execute that once, neatly working around the blocking issue.
It would be worthwhile to explore this route and build a POC binary that implements this functionality in order to see whether it is feasible at all and whether it allows us to reduce the overhead.