Refetch predefined variables if API cache is empty

What does this MR do and why?

For #381233 (closed)

Calculating the predefined variables to load in the form for manually creating pipeline is an expensive operation. When the frontend fetches ciConfigVariables, it is actually getting values stored in the API's cache.

The query can receive the following results:

• null: The cache is empty. This will trigger the backend to recalculate and populate the cache.
• []: No variables are defined through the CI configuration.
• [{...}]: Contains variables defined through the CI configuration.

This MR makes sure that the form polls the cache for updated values when there are no values stored in it yet (the first case). Ideally this should be determined through HTTP statuses, perhaps after #388810 (closed) is resolved.

In the previous implementation, we would stop polling after CONFIG_VARIABLES_TIMEOUT ms have passed (in this case, 5000, or 5 seconds). So when the frontend receives no variables from the cache after 5 seconds, we default to showing the form anyway without it (either an empty form or a form with variables pulled from the URL parameters, if they exist). This MR also makes sure we brought this behavior back, since it got removed when we migrated this component to use GraphQL.

Screenshots or screen recordings

Fetching from the cache

No polling since the values are already cached.

cache_populated

Polling when cache is empty

When switching to the dev branch, the query polls for updated variables. The loading icon shows up until the poll is done.

no_cache

Polling when cache is empty after CONFIG_VARIABLES_TIMEOUT (5 seconds) have passed

Since the cache is still empty after the timeout, the form stops polling and shows an empty form.

poll_timeout

How to set up and validate locally

1. Add the following to you CI config file:

   variables:
     DEPLOY_ENVIRONMENT:
       value: "canary"
       options:
         - "production"
         - "staging"
         - "canary"
       description: "The deployment target. Set to 'staging' by default."
     DEPLOY_NOTE:
       value: "Release"
       options:
         - "Release"
         - "Testing"
       description: "The reason why we're deploying."
   
   declare-config-vars:
     stage: test
     script:
       - echo $DEPLOY_NOTE
       - echo $DEPLOY_ENVIRONMENT

2. Go to the CI/CD > Pipelines > Run Pipeline (upper right button).
3. Check the network tab if polling works. If this is your first time opening the page, your variables will most likely not be cached yet.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.