Skip to content

Web-ui slow on large repo with thousands of directories and text files

Problem Statement

We are evaluating Gitlab (self-hosted) to move our internal repos to.

We have a large repo with about 2000 directories and 25,000 text files. These are small config files (no large binary files) so the total size on disk is small but it is a lot of files and directories. We edit these files using the built-in web editor (not the web IDE). From opening the repo in the browser and drilling down to a file to edit and opening the web editor to the point where you can start editing the file takes about 90 seconds. For someone having to wait 90 seconds every time they need to edit a file is not ideal.

I'm not sure if this is expected in terms of performance or if anything can be done to speed this up.

Our specs are

  • Self-hosted CE install on RHEL7 clone (VM)
  • gitlab-ce-12.4.2-ce.0.el7.x86_64
  • 8 CPU
  • 8GB mem
  • backend disks are SSD
  • 5 end-users using the web-ui

I've increased the unicorn workers to 13 according to the formula published in the install guide.

unicorn[‘worker_processes’] = 13

I’ve also tried to adjust the unicorn max memory

unicorn[‘worker_memory_limit_max’] = “2048 * 1 << 20”

I've also tried to increase the memory values for postgres (in /etc/gitlab/gitab.rb).

Are there any other tweaks or settings for repos with large numbers of directories and files?

So far the only thing I've been able to do that improves performance is to break up the repo into smaller repos. So instead of one repo with 2000 directories I created three repos and split the directories between them. With 1000 dirs it takes about 45 seconds. With 500 directories the time to edit a file comes down to about 30 seconds. Which is still a long time to wait to open a text editor.

Reach

Our end-users (User persona) are having this slowness issue.

10.0 = Impacts the vast majority (~80% or greater) of our users, prospects, or customers.

Impact

The impact is

2.0 = High impact

In it's current state of taking 90 seconds to edit a file in the web editor the system is frustrating to use.

Confidence

I'm hoping this is just a settings issue and can be tweaked to improve performance of the web-ui when dealing with a large repo.

80% = Medium confidence

Effort

Hopefully the effort to fix is minimal. Or if you can just let us know that for a repo with thousands of directories this is just how it is so we can reset our expectations.