Skip to content

Batch load CI/CD variables for 100+

What does this MR do and why?

To keep the same UX as what we currently have in REST, where we load all variables at once, I've decided to implement batch loading in graphQL. The way this is setup means that we fire off one query, then keep asking for more variables until we reach the end. Then, we show the entire variables list. This is because the API currently does not support sorting and by default, we sort values alphabetically. This meant that we were auto-sorting data as it loaded and so some new values that you loaded were now appearing above the scrolling point with an infinite scrolling style.

While we could have added pagination with actual pages, the problem is that again, because of the sorting, you'd get 20 items, sort these alphabetically, then load another 20 items and sort these alphabetically with no regards to the previous page. This was a very poor experience as well.

The last solution was disabling the sorting, but then we'd have a UX regression and users are now used to getting their values sorted alphabetically. Add to this that there are currently no filters/search... the best solution was batch loading.

There is currently an additional safeguard to avoid us accidentally creating an infinite loop, where we hard stop at 20 calls and show an error message + log to sentry. This would mean that we do not allow user to see more than 2000 variables, which seems pretty fair considering we want to limite variables to 200 soon.

Screenshots or screen recordings

Project variables

With flag off (REST) With flag on Before With flag on After
project_var_before project_var_before_graphql project_var_after

Group variables

With flag off (REST) With flag on Before With flag on After
group_var_before group_var_before_graphql group_var_after

Admin variables

With flag off (REST) With flag on Before With flag on After
instance_var_before instance_var_before_graphql instance_var_after

How to set up and validate locally

  • Navigate to Settings => CI/CD
  • Add more than 100 variables (Fun times 😅 ) I recommend using a little patch I made for project variables: hack_variables.patch which will generate 400 project variables! You can do the same for group, by adding the same logic but to the callGroupEndpoint and callAdminEndpoint
  • For the admin page, the limit to variable by default is 25. You need to manually change this value to 400 to test this out
  • Load the page
  • Notice that only 100 variables load
  • Scroll down do the end of the container
  • notice another query is fired and more variables are loaded

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #372728 (closed)

Edited by Frédéric Caplette

Merge request reports

Loading