Skip to content

Replace Wiki service list_pages by normal repository RPCs

Quang-Minh Nguyen requested to merge qmnguyen0711/357651-move-wiki-list_pages-method-to-gitaly into master

What does this MR do and why?

For #357651 (closed)

In &2862 (closed), we are trying to get rid of ruby sidecar inside Gitaly. The ruby sidecar is responsible for some hard-to-migrate when we invented Gitaly long before. One of a particular feature handled by Ruby sidecar is Wiki service. When GitLab Rails requests any Wiki-related data from Gitaly, it triggers a Wiki RPC call. This call is handled by Gitaly, but it instantly forwards to a Ruby gRPC server inside Gitaly. Eventually, the call is handled by Ruby. Underlying, the whole Wiki service is handled by Gollum. This gem structures Wiki data as a Git directory. Therefore, it makes a lot of sense to let Gitaly handles those business logic via normal repository RPCs.

This MR concentrates on moving the list_pages RPC. This RPC is responsible for fetching the list of all available pages. This call is replaced by the following RPCs:

  • repository#search_files_by_regexp: perform a search to fetch all files ending with eligible file extensions.
  • repository#blobs_at: fulfill the latest content of matched files above.
  • repository#last_commit_for_path: occasionally fetches the latest commit for path individually. This is a part of last updated date on the UI.

Those RPC replacements are not fast, but they don't make the situation worse. I did an in-depth analysis of the entry points of this RPC here: #357651 (comment 1077458464) to explain the reasons. Most of the use cases fetch a limited set of pages (20 pages maximum). The only exception is the Wiki APIs. They fetch all pages at once. I opened #371859 to follow up this performance issue.

This MR also removes the created_at sorting option on the UI in /-/wikis/pages pages. The reason behind this removal is mentioned in #357651 (comment 1081355714). This decision was also approved by @ericschurter, the product manager of ~"group::editor" - owner of Wiki features.

Screenshots or screen recordings

Before: the page includes sort options and sort direction

Screen_Shot_2022-09-06_at_00.19.10

After: the page includes the sort direction only

Screen_Shot_2022-09-06_at_00.13.47

Sort works with non-Ascii characters. Wiki pages of the same directory are grouped together

Screen_Shot_2022-09-06_at_00.19.57

Wiki API

Screen_Shot_2022-09-06_at_00.22.08

Wiki API with content

Screen_Shot_2022-09-06_at_00.22.37

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Quang-Minh Nguyen

Merge request reports