<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> *This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.* <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> Description copied from https://gitlab.com/gitlab-org/gitlab/issues/36865: Gitaly performs all read and write operations on Git data, right now that means on each (code) repository, design assets, and wikis. Soon this will include snippets too. All the different kinds of Git data use a shared interface, except for the Wiki data. Most of the wiki data is maintained through their own Gitaly service; the WikiService. This was done mostly to speed up the migration off NFS, as the risk of moving the code was limited to the actual move, and no additional change in logic. However, this makes the wikis the odd duck, which in my opinion should be mitigated. Other than just paying of technical debt, this has multiple benefits: 1. Developer productivity: * Given wikis, compared to code repositories, have much less features, porting features to wikis wouldn't require any Gitaly changes. 1. Performance improvements: * Gitaly has multiple performance improvements that greatly benefit latencies, such as: bitmap indexes, cat-file caches, repacking, garbage collection, and many more. 1. No dependency on Ruby+Rugged on the Gitaly side * Gollum, the gem being used to read/write the data, is written in Ruby. This potentially makes Gitaly-Ruby extra slow as Rugged does system calls while holding the GIL. Which in turn might mean each Wiki RPC is basically a noisy neighbour and slowing down their own requests as well as other clients requests. This could be partially be mitigated by using the gollum adapter that uses native Git, however this gem seems to have gone stale and unmaintained: https://rubygems.org/gems/gollum-grit_adapter ## Map of Wiki RPC calls - [Table of GitLab -> Gitaly -> Gollum calls](https://gitlab.com/gitlab-org/gitlab/issues/36865#note_258460795) - [Table of Gitaly Wiki RPC -> Proposed Gitaly non-Wiki RPC](https://gitlab.com/gitlab-org/gitlab/issues/36865#note_259194069) ## Draft proposal We should migrate RPC calls one by one. This could be done by writing a `gollum-gitaly_adapter` as proposed in the issue (https://gitlab.com/gitlab-org/gitlab/issues/36865) this Epic was created from, or through changes to `Gitlab::Git::Wiki` and other GitLab classes. ## Draft process Each piece of work would involve: 1. [Deprecating the Gitaly RPC](https://gitlab.com/gitlab-org/gitaly/blob/master/proto/DEPRECATION.md) 1. Release Gitaly 1. Replace the RPC in GitLab, using a Feature Flag to switch this on or off 1. Release GitLab 1. Monitor before and after 1. Remove Deprecated Gitaly RPC 1. Release Gitaly 1. Remove Feature Flag from GitLab 1. Release GitLab 1. Repeat! ## Definition of done The remaining Wiki Gollum methods are: - [x] [create_page](https://gitlab.com/gitlab-org/gitlab/blob/master/app/models/wiki.rb#L173) - [x] [update_page](https://gitlab.com/gitlab-org/gitlab/blob/master/app/models/wiki.rb#L186) - [x] [find_page](https://gitlab.com/gitlab-org/gitlab/blob/master/app/models/wiki.rb#L152) https://gitlab.com/gitlab-org/gitlab/-/issues/357649 - [x] [list_pages](https://gitlab.com/gitlab-org/gitlab/blob/master/app/models/wiki.rb#L126) https://gitlab.com/gitlab-org/gitlab/-/issues/357651 After we finished the deprecating process for all RPCs, we may need to perform some clean up tasks: - [x] Monitor and ensure that there is no requests to any of the target RPCS: - Client side: https://thanos.gitlab.net/graph?g0.expr=sum(rate(grpc_client_handled_total%7Bgrpc_method%3D~%22Wiki.*%22%7D%5B1h%5D))%20by%20(grpc_method)&g0.tab=0&g0.stacked=0&g0.range_input=2w&g0.max_source_resolution=0s&g0.deduplicate=1&g0.partial_response=0&g0.store_matches=%5B%5D&g0.step_input=3600 - Server side: https://thanos.gitlab.net/graph?g0.expr=sum(rate(grpc_server_handled_total%7Bgrpc_method%3D~%22Wiki.*%22%7D%5B1h%5D))%20by%20(grpc_method)&g0.tab=0&g0.stacked=0&g0.range_input=1w&g0.max_source_resolution=0s&g0.deduplicate=1&g0.partial_response=1&g0.store_matches=%5B%5D&g0.step_input=3600 - [x] Remove client calls in Rails code base - [x] Remove server handler in Gitaly