Frontend: GraphQL query times out with 503 when loading environment and variables
Problem
Attemping to load https://gitlab.com/gitlab-org/gitlab-foss/-/settings/ci_cd results in 503 timeout errors:
It looks like this GraphQL
query times out on occasion:
Kibana shows this SQL
timeout (https://log.gprd.gitlab.net/goto/5be38d60-5962-11ed-8d37-e9a2f393ea2a):
SELECT "environments".* FROM "environments" WHERE "environments"."project_id" = $1 ORDER BY "environments"."id" DESC LIMIT $2
This stems from this GraphQL
query:
[
{
"operationName": "getProjectEnvironments",
"variables": "[FILTERED]",
"query": "query getProjectEnvironments($fullPath: ID!) {
project(fullPath: $fullPath) {
id
environments {
nodes {
id
name
__typename
}
__typename
}
__typename
}
}
"
},
{
"operationName": "getProjectVariables",
"variables": "[FILTERED]",
"query": "query getProjectVariables($after: String, $first: Int = 100, $fullPath: ID!) {
project(fullPath: $fullPath) {
id
ciVariables(after: $after, first: $first) {
pageInfo {
...PageInfo
__typename
}
nodes {
...BaseCiVariable
environmentScope
masked
protected
__typename
}
__typename
}
__typename
}
}
fragment BaseCiVariable on CiVariable {
__typename
id
key
value
variableType
}
fragment PageInfo on PageInfo {
hasNextPage
hasPreviousPage
startCursor
endCursor
__typename
}
"
}
]
Backtrace
"exception.backtrace": [
"lib/gitlab/database/load_balancing/connection_proxy.rb:104:in `block in read_using_load_balancer'",
"lib/gitlab/database/load_balancing/load_balancer.rb:55:in `read'",
"lib/gitlab/database/load_balancing/connection_proxy.rb:103:in `read_using_load_balancer'",
"lib/gitlab/database/load_balancing/connection_proxy.rb:48:in `select_all'",
"lib/gitlab/graphql/pagination/keyset/connection.rb:135:in `block in limited_nodes'",
"lib/gitlab/utils/strong_memoize.rb:44:in `strong_memoize'",
"lib/gitlab/graphql/pagination/keyset/connection.rb:117:in `limited_nodes'",
"lib/gitlab/graphql/pagination/keyset/connection.rb:96:in `nodes'",
"lib/gitlab/graphql/connection_redaction.rb:23:in `block in nodes'",
"lib/gitlab/graphql/connection_redaction.rb:16:in `redacted'",
"lib/gitlab/graphql/connection_redaction.rb:23:in `nodes'",
"lib/gitlab/graphql/tracers/timer_tracer.rb:20:in `trace'",
"lib/gitlab/graphql/generic_tracing.rb:48:in `with_labkit_tracing'",
"lib/gitlab/graphql/generic_tracing.rb:38:in `platform_trace'",
"lib/gitlab/graphql/tracers/logger_tracer.rb:14:in `trace'",
"lib/gitlab/graphql/tracers/metrics_tracer.rb:13:in `trace'",
"lib/gitlab/graphql/tracers/application_context_tracer.rb:23:in `trace'",
"lib/gitlab/graphql/tracers/timer_tracer.rb:20:in `trace'",
"lib/gitlab/graphql/generic_tracing.rb:48:in `with_labkit_tracing'",
"lib/gitlab/graphql/generic_tracing.rb:38:in `platform_trace'",
"lib/gitlab/graphql/tracers/logger_tracer.rb:14:in `trace'",
"lib/gitlab/graphql/tracers/metrics_tracer.rb:13:in `trace'",
"lib/gitlab/graphql/tracers/application_context_tracer.rb:23:in `trace'",
"lib/gitlab/graphql/tracers/timer_tracer.rb:20:in `trace'",
"lib/gitlab/graphql/generic_tracing.rb:48:in `with_labkit_tracing'",
"lib/gitlab/graphql/generic_tracing.rb:38:in `platform_trace'",
"lib/gitlab/graphql/tracers/logger_tracer.rb:14:in `trace'",
"lib/gitlab/graphql/tracers/metrics_tracer.rb:13:in `trace'",
"lib/gitlab/graphql/tracers/application_context_tracer.rb:23:in `trace'",
"app/graphql/gitlab_schema.rb:51:in `multiplex'",
"app/controllers/graphql_controller.rb:160:in `execute_multiplex'",
"app/controllers/graphql_controller.rb:57:in `execute'",
"ee/lib/gitlab/ip_address_state.rb:10:in `with'",
UX Goal
The list appears in a dropdown in a modal on the project settings >> CI/CD >> Variables
section. Users should be able to easily discover an environment without having to scroll through lengthy lists or leaving the page.
Image |
---|
![]() |
Proposal
Show only the first set of elements (less than 100) and not paginate and rely on the search instead for users who have more to prevent encountering a 503 error.
NOTE: Environments scope will require a search bar and won't be paginated whereas variables will be paginated.
Pending: UX proposal for the experience.
Implementation Table
Group | Issue Link |
---|---|
backend | #385767 (closed) |
backend | #387803 (closed) |
frontend |
|
Implementation
header | header |
---|---|
#388952 (closed) / #394750 (closed) | Limit fetch/search for environments |
#388827 (closed) | Implement key-set pagination for CI Variables. |
#392947 / #393882 | Add search ability for CI Variables |
#392948 (closed) | Show confirmation toast message when adding, editing and deleting a CI variable. |
Rollout issue for Environment Scope changes: #395003 (closed)
Rollout issue for CI Variable Pagination: #392874