Performance and UX improvements to the environments page
While working on the environments search I noticed that the current environments page is not very well designed for performance when the project has a lot of environments. Counts of environments from gitlab.com:
gitlabhq_production=> SELECT project_id, COUNT(*)
FROM environments
GROUP BY project_id
ORDER BY 2 DESC
LIMIT 5;
project_id | count
------------+--------
xxx | 104686
xxx | 63919
xxx | 45138
xxx | 35704
xxx | 29861
Since we're moving this page to the graphql, we can do some small UI changes which would allow us to improve the performance of this page.
(in the worst case scenario, 3 queries below would take 20 seconds to load)
Available/stopped counters and folders counters:
Available/stopped really slow to fetch when we have 100k environments because Postgres basically need to visit every row belonging to project in the environments table(around 5s to get them)
We should add a limit to these counters and show 1000+
instead of the real number
Counters for folders are also hard to calculate for the same reason. But simply rounding this counters to 1000+ won't help here. But while we have these counters not rounded it will be harder to optimise this query.
To optimise it we'll need:
- cap the counter by 1000
- add index (something like
[project_id, state, COALESCE(environment_type, id::text), COALESCE(environment_type, name), id]
- use some counting hacks from https://charlesnagy.info/it/postgresql/group-by-limit-per-group-in-postgresql or something similar
Pagination
To show the number of pages and this 1/2/3 links we need to calculate the total number of folders which is also slow.