Skip to content

Performance and UX improvements to the environments page

While working on the environments search I noticed that the current environments page is not very well designed for performance when the project has a lot of environments. Counts of environments from gitlab.com:

gitlabhq_production=> SELECT project_id, COUNT(*) 
FROM environments  
GROUP BY project_id   
ORDER BY 2 DESC
LIMIT 5;
 project_id | count
------------+--------
   xxx  | 104686
   xxx |  63919
   xxx |  45138
   xxx |  35704
   xxx |  29861

Since we're moving this page to the graphql, we can do some small UI changes which would allow us to improve the performance of this page.

(in the worst case scenario, 3 queries below would take 20 seconds to load)

Available/stopped counters and folders counters:

Screenshot_2022-09-14_at_11.58.58

Available/stopped really slow to fetch when we have 100k environments because Postgres basically need to visit every row belonging to project in the environments table(around 5s to get them) We should add a limit to these counters and show 1000+ instead of the real number

Counters for folders are also hard to calculate for the same reason. But simply rounding this counters to 1000+ won't help here. But while we have these counters not rounded it will be harder to optimise this query.

To optimise it we'll need:

  1. cap the counter by 1000
  2. add index (something like [project_id, state, COALESCE(environment_type, id::text), COALESCE(environment_type, name), id]
  3. use some counting hacks from https://charlesnagy.info/it/postgresql/group-by-limit-per-group-in-postgresql or something similar

Pagination

Screenshot_2022-09-14_at_12.02.43

To show the number of pages and this 1/2/3 links we need to calculate the total number of folders which is also slow.

Edited by 🤖 GitLab Bot 🤖