Skip to content

Limit fetch and search queries for environments for CI Variables

Mireya Andres requested to merge ma/limit-env-variables-query into master

What does this MR do and why?

For #388952 (closed)

Due to the sheer amount of environments some projects have, the GraphQL query for fetching them will sometimes time out with 503s. Currently, we fetch all environments and filter the existing results on the frontend when the user uses the search field.

This MR builds the foundation for limiting the query to only fetch 20 environments for project environments, which will hopefully lessen the load of our requests. The user can use the search bar in the dropdown to narrow down the query, but this also limits the results to 20 at a time. The user can refine their search if they want to see more results. The search bar now refetches the GraphQL query, instead of filtering on the frontend.

All changes are under the ci_limit_environment_scope feature flag.

NOTE: We don't have a dedicated query for fetching group environment scopes yet. Development for this will continue once #394750 (closed) is done. After that, we can enable the feature flag. (Or maybe we can release this for projects only for the moment? 🤔)

Screenshots or screen recordings

Click to expand
Before After
ff_disabled project_env_scope
Initial Request Search Request
initial_query search_query

Experience for group environment scopes should not be affected:

group_env_scope

How to set up and validate locally

Setup

  1. Enable the ci_limit_environment_scope feature flag.
    • Disable the ci_variables_pages feature flag if you have this enabled. This is the feature flag for enabling pagination for the variables table, however there is an issue that causes the pagination to limit the environment scopes for group variables. This will be unblocked by #394750 (closed).
  2. Use the rake tasks in !113172 (merged) to seed your project with environments. Make sure you have more than 20 environments. For example:
      bundle exec rake "gitlab:seed:project_environments[path/project, 10, dev_env_]"
      bundle exec rake "gitlab:seed:project_environments[path/project, 10, staging_env_]"

Verification

  1. Go to Settings > CI/CD > and expand the Variables section. Click on the button to create a variable (bottom of the table) to open the modal.
  2. The dropdown for Environment Scopes should now only fetch 20 environments (check Network tab).
  3. When using the search bar in the dropdown, this also queries for at most 20 environments.
  4. Changes should not affect the modal for group variables. ALL environment scopes will still be available in the group modal, and the search is done in the frontend instead of firing a search request.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Mireya Andres

Merge request reports